arXiv Machine Learning
TestBED: An Open-Source Toolkit for Bayesian Experimental Design
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jul 21
Deep Adaptive Bayesian Screening
arXiv:2607. 16927v1 Announce Type: cross Abstract: We introduce Deep Adaptive Bayesian Screening (DABS), a method for performing adaptive factorial screening in high-dimensional discrete design spaces.
arXiv AI
Aug 3
Identifying Informative Environments for Cognition Parameter Inference via Bayesian Experimental Design
arXiv:2607. 28894v1 Announce Type: new Abstract: Computational cognitive modeling seeks to infer latent cognitive mechanisms underlying observed behavior.
arXiv Machine Learning
Oct 19
Robust Experimental Design via Generalised Bayesian Inference
The paper introduces Generalised Bayesian Optimal Experimental Design (GBOED), extending generalised Bayesian inference to experimental design. It replaces the likelihood in Bayesian updates with a loss function, deriving a new acquisition function called Gibbs expected information gain. Empirical results show that GBOED improves robustness against outliers and mis-specified outcome noise distributions.
arXiv AI
Jul 7
JADAI: Jointly Amortizing Adaptive Design and Bayesian Inference
arXiv:2512. 22999v2 Announce Type: replace-cross Abstract: We consider problems of parameter estimation where design variables can be actively optimized to maximize information gain.
arXiv AI
Oct 28
BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks
BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.
arXiv AI
Oct 16
Amortizing Sequential Experimental Design in Reasoning LLMs
The paper proposes a method to embed sequential Bayesian experimental design (BED) into large language models (LLMs) by training the models to reason about action selection using information-gain rewards. By specifying the experimental generative model separately from the LLM’s beliefs, the authors optimize expected information gain under a chosen model. Experiments on the Twenty Questions task show that the resulting policies outperform both inference- and training-time BED baselines while incurring lower computational costs, and the approach generalizes to unseen categories and transfers positively to the Medi-Q clinical dialogue benchmark.