arXiv AI

Goal-Oriented Sequential Bayesian Experimental Design for Causal Learning

GO-CBED is a goal‑oriented Bayesian framework for sequential causal experimental design that directly maximizes expected information gain on user‑specified causal quantities, rather than inferring the full causal model. It employs a variational lower bound estimator optimized with a transformer‑based policy network and normalizing flow posteriors, enabling real‑time, non‑myopic decision making. Experiments on synthetic structural causal models and semi‑synthetic gene regulatory networks show that GO‑CBED consistently outperforms existing baselines, especially under limited experimental budgets and complex causal mechanisms.

arXiv AI
Jul 14

CDFM: Towards a General-Purpose Causal Discovery Foundation Model

arXiv:2607. 11508v1 Announce Type: cross Abstract: Causal discovery, the process of recovering underlying causal structures from observational data, is a fundamental pursuit across scientific disciplines.

By Jie Qiao, Ruichu Cai, Zijian Li, Weilin Chen, Pengfei Hua, Boyan Xu, Zhengming Chen, Zhifeng Hao, Peng Cui
arXiv AI
Oct 16

Amortizing Sequential Experimental Design in Reasoning LLMs

The paper proposes a method to embed sequential Bayesian experimental design (BED) into large language models (LLMs) by training the models to reason about action selection using information-gain rewards. By specifying the experimental generative model separately from the LLM’s beliefs, the authors optimize expected information gain under a chosen model. Experiments on the Twenty Questions task show that the resulting policies outperform both inference- and training-time BED baselines while incurring lower computational costs, and the approach generalizes to unseen categories and transfers positively to the Medi-Q clinical dialogue benchmark.

By Anita Kriz, Antonin Berthon, Nicol\'as Astorga, Mihaela van der Schaar
arXiv Machine Learning
Aug 19

TabCausal: Pretraining Across Causal Environments for Tabular Causal Discovery

TabCausal is a causal discovery foundation model that learns to map datasets directly to causal graphs by pretraining across diverse causal environments. It uses a dynamic task construction strategy to expose the model to varied graph priors, mechanisms, noise models, dimensions, sample sizes, and intervention regimes, improving transferability from observational and mixed‑interventional data. On large synthetic benchmarks and a new protocol‑guided semantic benchmark, TabCausal outperforms many classical baselines and shows robust structure recovery, especially when interventional evidence is available.

By Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang, Han-Jia Ye
arXiv Machine Learning
Oct 23

ACTIVA: Amortized Causal Effect Estimation via Variational Autoencoders

ACTIVA is a transformer‑based conditional variational autoencoder designed to estimate full interventional distributions from observational data without requiring interventional data, a causal graph, or per‑task fitting. It learns a conditional latent prior that amortizes causal knowledge across diverse training tasks, enabling a single forward pass to handle new causal models drawn from the training distribution. Empirical results on synthetic and biologically realistic gene‑expression simulations show that ACTIVA outperforms correlational baselines, reduces spurious non‑descendant effects, and matches strong amortized baselines, while also validating its theoretical mixture‑of‑distributions property on observationally equivalent models.

By Andreas Sauter, Saber Salehkaleybar, Frank van Harmelen, Aske Plaat, Erman Acar
arXiv Statistics ML
2d ago

Action-BED: Task-Driven Bayesian Experimental Design with Singly Intractable Objectives

The paper introduces ACTION-BED, a Bayesian experimental design framework that replaces traditional uncertainty‑reduction objectives with expected future loss (EFL) on downstream actions. By reformulating EFLs into singly intractable objectives, ACTION-BED enables joint optimisation of design and action policies using stochastic gradients without explicit posterior or marginal likelihood estimation. This approach simplifies learning of design policies and allows easy customization to various downstream tasks and loss functions.

By Tom Rossa, Angus Phillips, Tom Rainforth
arXiv AI
Oct 26

A Foundation Model for Policy In-Context Longitudinal Causal Effect Estimation

The paper introduces PICLE-PFN, a foundation model for longitudinal causal effect estimation that handles dynamic treatment policies. It represents policies via in-context history-decision demonstrations and is pretrained on synthetic data from longitudinal structural causal models, enabling it to predict conditional average potential outcomes in a single forward pass without task-specific training. Experiments on synthetic and semi-synthetic benchmarks show competitive RMSE and better uncertainty calibration while reducing deployment runtime compared to conventional estimators.

By Wenxin Chen, Kyra Gan, Mingyu Lu, Zilin Jing, Fei Wang, Hoifung Poon, Tristan Naumann, Peniel Argaw
arXiv AI
Jul 7

Amortising Bayesian Experimental Design for Sequential Information Gathering in LLMs

arXiv:2607. 03426v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit strong reasoning and world-knowledge capabilities, yet often struggle to gather information effectively across the multi-turn interactions required in sequential decision-making settings.

By Jakob Hartmann, James Harvey, Jhonathan Navott, Erik Y. Wang, Luckeciano C. Melo, Flaviu Cipcigan, Cheng Zhang, Alessandro Abate