← Back to all news
arXiv Machine Learning By Sahel Iqbal, Tom Rainforth

Designing Experiments for External Decision Makers

Read the original on arXiv Machine Learning →

The paper introduces usage‑aware experimental design, a framework that extends standard Bayesian experimental design to settings where external decision makers act on collected data. It trains adaptive design policies to minimise the experimenter’s expected loss under external rules, which may be non‑Bayesian or black‑box. The authors show that more informative data can be harmful under such rules, so usage risk cannot be expressed as expected uncertainty reduction, thereby generalising standard BED.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 19

Robust Experimental Design via Generalised Bayesian Inference

The paper introduces Generalised Bayesian Optimal Experimental Design (GBOED), extending generalised Bayesian inference to experimental design. It replaces the likelihood in Bayesian updates with a loss function, deriving a new acquisition function called Gibbs expected information gain. Empirical results show that GBOED improves robustness against outliers and mis-specified outcome noise distributions.

By Yasir Zubayr Barlas, Sabina J. Sloman, Samuel Kaski
More like this →
arXiv Machine Learning
Sep 29

Offline Policy Evaluation as a decision support tool for designing Adaptive Experiments

The paper explores how data from fixed A/B tests can guide the deployment of adaptive experiments using contextual bandits. By combining off‑policy evaluation with a controlled warm‑start simulation, the authors rank pre‑specified adaptive and non‑adaptive policies using doubly robust estimators. Experiments on synthetic trials and real benchmarks show that adaptive, context‑aware policies outperform fixed allocations when heterogeneity exists, but offer little advantage otherwise.

By Jo\~ao Victor Ferreira Alves, Eduardo Rocha Laurentino, Gustavo de Oliveira Kanno, Thiago Costa Rizuti da Rocha
benchmarks
More like this →
arXiv AI
Jun 15

CARE: Controlling LLM-Generated Policies through Auditable Review of Evidence in Scientific Experimentation

arXiv:2606. 14581v1 Announce Type: cross Abstract: Granting LLMs direct control over costly, irreversible scientific experiments leads to unsafe exploration and unstable performance, but discarding LLM creativity entirely sacrifices significant optimization potential.

By Guanyu Liu, Weiyi Kong, Zeyu Wang, Boer Zhang, Baiqing Li, Peiyu Zhang, Tianyu Shi
llmsefficiencybenchmarks
More like this →
arXiv Machine Learning
Jul 10

Bayesian Experimental Design via Score Matching

arXiv:2607. 08335v1 Announce Type: cross Abstract: Policy-based approaches to Bayesian experimental design (BED) allow the learning of deep policy networks that adaptively make intelligent design decisions based on previously collected data.

By Angus Phillips, Gavin Kerrigan, Tom Rainforth
More like this →
arXiv Machine Learning
Jul 21

Deep Adaptive Bayesian Screening

arXiv:2607. 16927v1 Announce Type: cross Abstract: We introduce Deep Adaptive Bayesian Screening (DABS), a method for performing adaptive factorial screening in high-dimensional discrete design spaces.

By Jade Lejeune Herman, Arno Strouwen, Johan A. K. Suykens, Peter Goos
efficiencybenchmarks
More like this →