arXiv Machine Learning By Manuel H\"ulskamp, Julius Kobialka, Emanuel Sommer, Thomas Nagler, David R\"ugamer

How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?

Read the original on arXiv Machine Learning →

The paper investigates how well Prior-Data Fitted Networks (PFNs) approximate Bayesian posterior predictive distributions (PPDs). By decomposing deployment error into prior, context, and amortization components, the authors find that PFNs generally recover the target PPD accurately, but residual amortization errors remain in rare settings, leading to disagreement among independently pretrained models. They demonstrate that Bayesian model averaging can consistently reduce this uncertainty and improve predictions, even matching full-size TabICLv2 models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
3d ago

SelectPFN: Amortized Bayesian Variable Selection for Sparse Nonlinear Regression with Prior-Data Fitted Networks

SelectPFN is a prior‑data‑fitted network that jointly performs Bayesian variable selection and prediction for sparse nonlinear regression. In a single forward pass it outputs posterior inclusion probabilities for each variable and makes predictions using only sampled subsets, ensuring excluded variables cannot influence the result. The method achieves exact finite‑sample false discovery rate control via column‑permutation equivariance and knockoff statistics, and outperforms lasso, BART, DART, LassoNet, and other sparse neural networks on synthetic and real data with up to 500 predictors.

By Bin Luo
arXiv Machine Learning
3d ago

Amortized Factor Inference Networks for Posterior Inference

The paper introduces Amortized Factor Inference Networks (AFINs), a single inference network that can generalize across varying priors, likelihoods, latent dimensions, and observation counts. AFINs represent a model as a list of typed prior and likelihood factors and use an encode–merge–decode architecture to produce a Gaussian posterior approximation, with learned parameters independent of latent dimensionality or observation count. Experiments on synthetic and real datasets demonstrate that a single trained AFIN achieves competitive posterior approximations at significantly lower test‑time cost than NUTS and per‑task variational inference, with optional importance sampling further improving accuracy.

By Joohwan Ko, Justin Domke