arXiv Machine Learning

Amortized Factor Inference Networks for Posterior Inference

The paper introduces Amortized Factor Inference Networks (AFINs), a single inference network that can generalize across varying priors, likelihoods, latent dimensions, and observation counts. AFINs represent a model as a list of typed prior and likelihood factors and use an encode–merge–decode architecture to produce a Gaussian posterior approximation, with learned parameters independent of latent dimensionality or observation count. Experiments on synthetic and real datasets demonstrate that a single trained AFIN achieves competitive posterior approximations at significantly lower test‑time cost than NUTS and per‑task variational inference, with optional importance sampling further improving accuracy.

arXiv Machine Learning
Sep 1

Delta-AI: Local objectives for amortized inference in sparse graphical models

arXiv:2310.02423v3 Announce Type: replace Abstract: We present a new algorithm for amortized inference in sparse probabilistic graphical models (PGMs), which we call $\Delta$-amortized inference ($\D...

By Jean-Pierre Falet, Hae Beom Lee, Esmeralda S. Whitammer, Chen Sun, Dragos Secrieru, Thomas Jiralerspong, Dinghuai Zhang, Guillaume Lajoie, Yoshua Bengio
arXiv Machine Learning
Oct 12

How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?

The paper investigates how well Prior-Data Fitted Networks (PFNs) approximate Bayesian posterior predictive distributions (PPDs). By decomposing deployment error into prior, context, and amortization components, the authors find that PFNs generally recover the target PPD accurately, but residual amortization errors remain in rare settings, leading to disagreement among independently pretrained models. They demonstrate that Bayesian model averaging can consistently reduce this uncertainty and improve predictions, even matching full-size TabICLv2 models.

By Manuel H\"ulskamp, Julius Kobialka, Emanuel Sommer, Thomas Nagler, David R\"ugamer