arXiv Machine Learning

TestBED: An Open-Source Toolkit for Bayesian Experimental Design

arXiv Machine Learning
Jul 21

Deep Adaptive Bayesian Screening

arXiv:2607. 16927v1 Announce Type: cross Abstract: We introduce Deep Adaptive Bayesian Screening (DABS), a method for performing adaptive factorial screening in high-dimensional discrete design spaces.

By Jade Lejeune Herman, Arno Strouwen, Johan A. K. Suykens, Peter Goos
arXiv Machine Learning
Oct 19

Robust Experimental Design via Generalised Bayesian Inference

The paper introduces Generalised Bayesian Optimal Experimental Design (GBOED), extending generalised Bayesian inference to experimental design. It replaces the likelihood in Bayesian updates with a loss function, deriving a new acquisition function called Gibbs expected information gain. Empirical results show that GBOED improves robustness against outliers and mis-specified outcome noise distributions.

By Yasir Zubayr Barlas, Sabina J. Sloman, Samuel Kaski
arXiv AI
Oct 28

BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.

By Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra, Bryan Kian Hsiang Low
arXiv AI
Oct 16

Amortizing Sequential Experimental Design in Reasoning LLMs

The paper proposes a method to embed sequential Bayesian experimental design (BED) into large language models (LLMs) by training the models to reason about action selection using information-gain rewards. By specifying the experimental generative model separately from the LLM’s beliefs, the authors optimize expected information gain under a chosen model. Experiments on the Twenty Questions task show that the resulting policies outperform both inference- and training-time BED baselines while incurring lower computational costs, and the approach generalizes to unseen categories and transfers positively to the Medi-Q clinical dialogue benchmark.

By Anita Kriz, Antonin Berthon, Nicol\'as Astorga, Mihaela van der Schaar
arXiv AI
Oct 23

From Prompts to Posteriors: Can LLM Agents Perform Bayesian Inference in Context?

The paper investigates whether large language models (LLMs) can perform principled Bayesian inference when updating beliefs in context. By creating an evaluation framework that includes qualitative epistemological tests, quantitative benchmarks, and analyses of downstream impacts, the authors find that LLMs typically violate Bayes' rule but that their performance improves dramatically when equipped with advanced reasoning and code‑execution tools, allowing them to approximate analytically intractable posteriors. The study suggests that appropriately tool‑augmented LLMs could already be leveraged to accelerate scientific discovery through complex belief updates.

By Linus Filbry, Luckeciano C. Melo, Yarin Gal