The paper investigates how well Prior-Data Fitted Networks (PFNs) approximate Bayesian posterior predictive distributions (PPDs). By decomposing deployment error into prior, context, and amortization components, the authors find that PFNs generally recover the target PPD accurately, but residual amortization errors remain in rare settings, leading to disagreement among independently pretrained models. They demonstrate that Bayesian model averaging can consistently reduce this uncertainty and improve predictions, even matching full-size TabICLv2 models.
By Manuel H\"ulskamp, Julius Kobialka, Emanuel Sommer, Thomas Nagler, David R\"ugamer
ProximalFM is a transformer‑based model that uses prior‑data fitted networks (PFNs) to perform Bayesian proximal causal inference under hidden confounding. By training on synthetic data generated from structural causal models with oracle counterfactuals, it amortizes the Bayesian operator inversion into a single forward pass, producing posterior estimates of the conditional average treatment effect (CATE). The approach consistently outperforms prior methods across various proximal regimes, especially when latent confounding is strong and proxy variables are weakly informative, and it requires no dataset‑specific tuning.
By Christophe Muller, Ayub Kharel, Alex Luedtke, Chan Park, Eric Tchetgen Tchetgen, Juan L. Gamella, Rahul Krishnan, Ricardo Silva, Jakob Zeitler
arXiv:2607. 05806v1 Announce Type: new Abstract: Training data for machine learning is routinely collected by a selection process the model never sees: loans are observed only when granted, outcomes only when a test was ordered.
By Gunner Levi Howe
arXiv:2603. 12037v2 Announce Type: replace Abstract: Foundation models based on prior-data fitted networks (PFNs) have shown strong empirical performance in causal inference by framing the task as an in-context learning problem.
By Valentyn Melnychuk, Vahid Balazadeh, Stefan Feuerriegel, Rahul G. Krishnan
The paper investigates feature priming in high‑dimensional online linear regression, showing that estimating feature weights from past data and refitting a minimum‑norm predictor can lead to regret that scales with sparsity rather than ambient dimension. It provides a negative answer to a COLT 2023 open problem by proving that three natural priming rules incur ≥Ω(min{T,√d}) regret against a zero‑loss one‑sparse comparator, due to cheap nuisance interpolation that underweights truly predictive coordinates. The authors also identify conditions under which regret is governed by data rank and present constructions that achieve tight univariate rates, while noting that the multivariate case remains unresolved.
By Huibo Xu, Shi Fu, Qixin Zhang, Dacheng Tao
arXiv:2601. 07944v2 Announce Type: replace-cross Abstract: Since the turn of the century, approximate Bayesian inference has steadily evolved as new computational techniques have been incorporated to handle increasingly complex, large-scale predictive problems.
By Roy Shivam Ram Shreshtth, Arnab Hazra, Gourab Mukherjee
arXiv:2605. 08446v3 Announce Type: replace Abstract: Bayesian neural networks are typically trained against the evidence lower bound (ELBO), whose Jensen gap closes only when the variational posterior is exact.
By Pavel Prochazka
arXiv:2609.36883v1 Announce Type: new
Abstract: Tabular foundation models (TFMs) are increasingly popular because they deliver strong predictions on new datasets through in-context learning, without...
By Tianqi Zhao, Tianyi Zhuang, Shuo Duan, Guanyang Wang, Yan Shuo Tan, Qiong Zhang
IQS-BO introduces an in‑context query selection method for Bayesian optimisation that learns to choose evaluation points via supervised learning on synthetic priors. The approach uses a Prior‑data Fitted Network (PFN) to predict, in a single forward pass, the probability that each candidate maximises the objective, thereby amortising the decision step and eliminating the need for costly surrogate refitting or acquisition maximisation. Experiments show that IQS‑BO matches or surpasses traditional Gaussian‑process‑based BO and other in‑context methods on both synthetic and real‑world benchmarks, and that a mixture prior combining GP samples with challenging function types further improves performance.
By Luca Geminiani, Nadja Klein
arXiv:2609.06941v1 Announce Type: new
Abstract: Causal effect estimation asks how an outcome would change under an intervention, and medicine, economics, and public policy all treat it as a foundatio...
By Haohao Zhou
Hierarchical data is ubiquitous in the empirical sciences and is most commonly analyzed with generalized linear mixed-effects models (GLMMs). Bayesian inference for GLMMs yields calibrated uncertainty...
arXiv:2609.24422v1 Announce Type: new
Abstract: Hierarchical data is ubiquitous in the empirical sciences and is most commonly analyzed with generalized linear mixed-effects models (GLMMs). Bayesian...
By Alex Kipnis, Marcel Binz, Eric Schulz