arXiv Machine Learning

Prior-Amortized In-Context Bayesian Inference for Generalized Linear Mixed-Effects Models

arXiv Machine Learning
Oct 7

ProximalFM: Amortized Proximal Causal Inference under Hidden Confounding

ProximalFM is a transformer‑based model that uses prior‑data fitted networks (PFNs) to perform Bayesian proximal causal inference under hidden confounding. By training on synthetic data generated from structural causal models with oracle counterfactuals, it amortizes the Bayesian operator inversion into a single forward pass, producing posterior estimates of the conditional average treatment effect (CATE). The approach consistently outperforms prior methods across various proximal regimes, especially when latent confounding is strong and proxy variables are weakly informative, and it requires no dataset‑specific tuning.

By Christophe Muller, Ayub Kharel, Alex Luedtke, Chan Park, Eric Tchetgen Tchetgen, Juan L. Gamella, Rahul Krishnan, Ricardo Silva, Jakob Zeitler
arXiv Machine Learning
Oct 19

ENFR: Evidence Networks with Flow-Matched References for Amortized Simulation-Based Model Inference

The paper introduces ENFR, a method for estimating the absolute evidence ζ of a data‑generating model in Bayesian inference. It learns Bayes factors relative to a flow‑matched reference and then recovers the absolute evidence, improving over existing flow‑based baselines across multiple applications. The authors also provide a joint architecture for estimating evidences of multiple models and demonstrate its performance and scalability on diverse scientific datasets.

By Kai Lehman, Jameson Dong, Carolina Cuesta-Lazaro, Niall Jeffrey, Benjamin D. Wandelt, Francisco Villaescusa-Navarro, Sven Krippendorf, Jochen Weller
arXiv Machine Learning
Oct 16

Amortized Factor Inference Networks for Posterior Inference

The paper introduces Amortized Factor Inference Networks (AFINs), a single inference network that can generalize across varying priors, likelihoods, latent dimensions, and observation counts. AFINs represent a model as a list of typed prior and likelihood factors and use an encode–merge–decode architecture to produce a Gaussian posterior approximation, with learned parameters independent of latent dimensionality or observation count. Experiments on synthetic and real datasets demonstrate that a single trained AFIN achieves competitive posterior approximations at significantly lower test‑time cost than NUTS and per‑task variational inference, with optional importance sampling further improving accuracy.

By Joohwan Ko, Justin Domke
arXiv Machine Learning
Sep 1

Delta-AI: Local objectives for amortized inference in sparse graphical models

arXiv:2310.02423v3 Announce Type: replace Abstract: We present a new algorithm for amortized inference in sparse probabilistic graphical models (PGMs), which we call $\Delta$-amortized inference ($\D...

By Jean-Pierre Falet, Hae Beom Lee, Esmeralda S. Whitammer, Chen Sun, Dragos Secrieru, Thomas Jiralerspong, Dinghuai Zhang, Guillaume Lajoie, Yoshua Bengio
arXiv Machine Learning
Oct 28

Adapting Neural Posterior Estimators for Simulation-Based Inference in Agent-Based Models

The paper presents a framework called SNPE‑TTT for adapting pre‑trained neural posterior estimators, specifically conditional normalizing flows, to new inference tasks in agent‑based models (ABMs). It explores four adaptation strategies—full‑parameter adaptation, low‑rank adaptation, and two gradient‑based update spaces—and evaluates them on economic simulation models with both tractable and intractable likelihoods. Results show that the framework can improve posterior recovery with a limited number of target simulations, though performance varies by task and adaptation method.

By Junyao Zhang, Jinglai Li, Junqi Tang
arXiv Machine Learning
Oct 23

ACTIVA: Amortized Causal Effect Estimation via Variational Autoencoders

ACTIVA is a transformer‑based conditional variational autoencoder designed to estimate full interventional distributions from observational data without requiring interventional data, a causal graph, or per‑task fitting. It learns a conditional latent prior that amortizes causal knowledge across diverse training tasks, enabling a single forward pass to handle new causal models drawn from the training distribution. Empirical results on synthetic and biologically realistic gene‑expression simulations show that ACTIVA outperforms correlational baselines, reduces spurious non‑descendant effects, and matches strong amortized baselines, while also validating its theoretical mixture‑of‑distributions property on observationally equivalent models.

By Andreas Sauter, Saber Salehkaleybar, Frank van Harmelen, Aske Plaat, Erman Acar
arXiv Machine Learning
Oct 12

How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?

The paper investigates how well Prior-Data Fitted Networks (PFNs) approximate Bayesian posterior predictive distributions (PPDs). By decomposing deployment error into prior, context, and amortization components, the authors find that PFNs generally recover the target PPD accurately, but residual amortization errors remain in rare settings, leading to disagreement among independently pretrained models. They demonstrate that Bayesian model averaging can consistently reduce this uncertainty and improve predictions, even matching full-size TabICLv2 models.

By Manuel H\"ulskamp, Julius Kobialka, Emanuel Sommer, Thomas Nagler, David R\"ugamer