arXiv:2603. 12037v2 Announce Type: replace Abstract: Foundation models based on prior-data fitted networks (PFNs) have shown strong empirical performance in causal inference by framing the task as an in-context learning problem.
By Valentyn Melnychuk, Vahid Balazadeh, Stefan Feuerriegel, Rahul G. Krishnan
ProximalFM is a transformer‑based model that uses prior‑data fitted networks (PFNs) to perform Bayesian proximal causal inference under hidden confounding. By training on synthetic data generated from structural causal models with oracle counterfactuals, it amortizes the Bayesian operator inversion into a single forward pass, producing posterior estimates of the conditional average treatment effect (CATE). The approach consistently outperforms prior methods across various proximal regimes, especially when latent confounding is strong and proxy variables are weakly informative, and it requires no dataset‑specific tuning.
By Christophe Muller, Ayub Kharel, Alex Luedtke, Chan Park, Eric Tchetgen Tchetgen, Juan L. Gamella, Rahul Krishnan, Ricardo Silva, Jakob Zeitler
The paper investigates how well Prior-Data Fitted Networks (PFNs) approximate Bayesian posterior predictive distributions (PPDs). By decomposing deployment error into prior, context, and amortization components, the authors find that PFNs generally recover the target PPD accurately, but residual amortization errors remain in rare settings, leading to disagreement among independently pretrained models. They demonstrate that Bayesian model averaging can consistently reduce this uncertainty and improve predictions, even matching full-size TabICLv2 models.
By Manuel H\"ulskamp, Julius Kobialka, Emanuel Sommer, Thomas Nagler, David R\"ugamer
SelectPFN is a prior‑data‑fitted network that jointly performs Bayesian variable selection and prediction for sparse nonlinear regression. In a single forward pass it outputs posterior inclusion probabilities for each variable and makes predictions using only sampled subsets, ensuring excluded variables cannot influence the result. The method achieves exact finite‑sample false discovery rate control via column‑permutation equivariance and knockoff statistics, and outperforms lasso, BART, DART, LassoNet, and other sparse neural networks on synthetic and real data with up to 500 predictors.
By Bin Luo
arXiv:2310.02423v3 Announce Type: replace
Abstract: We present a new algorithm for amortized inference in sparse probabilistic graphical models (PGMs), which we call $\Delta$-amortized inference ($\D...
By Jean-Pierre Falet, Hae Beom Lee, Esmeralda S. Whitammer, Chen Sun, Dragos Secrieru, Thomas Jiralerspong, Dinghuai Zhang, Guillaume Lajoie, Yoshua Bengio
The paper introduces a model‑agnostic inference framework for partially identified causal effects that leverages covariate information without requiring discrete covariates or accurate conditional distribution estimates. Using duality theory for optimal transport, the method delivers uniformly valid inference in randomized experiments, is doubly robust in observational settings, achieves asymptotic unbiasedness when nuisance parameters converge semiparametrically, and allows multiplier‑bootstrap selection of covariates and models while remaining computationally efficient. Empirical applications show the approach consistently narrows identified sets and confidence intervals without imposing extra structural assumptions.
By Wenlong Ji, Lihua Lei, Asher Spector