arXiv Machine Learning

tidyHEBO: Robust General-Purpose Bayesian Optimization with Model-Consistent Warping and Pareto Search

tidyHEBO is a BoTorch-native Bayesian optimization tool that jointly applies Yeo-Johnson output warping to a Gaussian‑process surrogate, evaluates acquisition functions on the original objective scale, and conducts constrained cumulative Pareto search across multiple acquisition criteria. Using only default settings, it outperformed other methods on the Olympus benchmark and performed strongly on synthetic, Needle‑in‑a‑Haystack, and Bayesmark tasks, while adaptive batching offered a trade‑off between parallelization and optimization quality. These results position tidyHEBO as a robust, reproducible optimizer suitable for diverse practical problems, including scientific applications and hyperparameter tuning.

arXiv Machine Learning
Jul 14

Modernizing HEBO: a robust Bayesian optimization baseline for practical heteroskedastic and non-stationary problems

arXiv:2607. 10669v1 Announce Type: new Abstract: Bayesian optimization is increasingly used to guide data-efficient experimentation in chemistry, materials science, and related laboratory settings, but its practical performance depends strongly on how well surrogate-model assumptions match the geometry and noise structure of the underlying objective.

By L. A. Zhukov, E. V. Shaburova, D. V. Antonets
arXiv Machine Learning
Sep 4

No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels

The paper introduces Finite-Library Input-Warped Bayesian Optimization (FLIWBO), a method that selects input warps from a finite library to adapt the geometry used by Gaussian‑process Bayesian optimization. FLIWBO maintains high‑probability convergence guarantees while improving sample efficiency on problems where raw coordinates poorly match the objective’s geometry, such as log‑scaled hyperparameters or localized peaks. Experiments on synthetic benchmarks, Fashion‑MNIST hyperparameter tuning, and a 20‑dimensional multi‑agent system design demonstrate that FLIWBO‑UCB outperforms raw‑coordinate GP‑UCB and other methods with regret guarantees, especially under misspecified geometry.

By Edvin Ketabati Augustinsson, Robert A. Bridges
arXiv Statistics ML
Oct 2

IQS-BO: In-Context Query Selection for Bayesian Optimisation

IQS-BO introduces an in‑context query selection method for Bayesian optimisation that learns to choose evaluation points via supervised learning on synthetic priors. The approach uses a Prior‑data Fitted Network (PFN) to predict, in a single forward pass, the probability that each candidate maximises the objective, thereby amortising the decision step and eliminating the need for costly surrogate refitting or acquisition maximisation. Experiments show that IQS‑BO matches or surpasses traditional Gaussian‑process‑based BO and other in‑context methods on both synthetic and real‑world benchmarks, and that a mixture prior combining GP samples with challenging function types further improves performance.

By Luca Geminiani, Nadja Klein
arXiv AI
Oct 28

BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.

By Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra, Bryan Kian Hsiang Low
arXiv Machine Learning
Aug 3

Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery

arXiv:2607. 29225v1 Announce Type: new Abstract: Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is rarely evaluated alongside optimization performance.

By Panagiotis Krokidas, Christoforos Rekatsinas, Vassilis Sioros, Grigorios M. Chatziathanasiou, Efi-Maria Papia, George Giannakopoulos
arXiv Statistics ML
Nov 9

Simple Extensions of Single-Objective Acquisition Functions and Hedge Strategies for Multi-Objective Bayesian Optimization

The paper proposes a simple framework that extends standard single-objective Bayesian optimization acquisition functions to the multi-objective setting using a hypervolume-based approach, and adapts hedge strategies for multi-objective use. This method requires minimal changes to existing pipelines and preserves the intuitive interpretation and computational efficiency of the original strategies. Experiments on synthetic and real-world benchmarks show that these extensions consistently match or outperform more complex state-of-the-art multi-objective Bayesian optimization methods.

By Haris Moazam Sheikh