The paper introduces fast Bayesian optimization (BO) methods that leverage linear structures in existing BO algorithms, enabling efficient optimization across a wide range of problem sizes—from 10 to over 6000 dimensions. These new methods achieve performance comparable to state‑of‑the‑art BO while being 100 to 1000 times faster, thereby expanding BO’s applicability to problems that were previously too time‑intensive. The authors also analyze the speed‑performance Pareto frontier, showing that many regions remain unfilled by current algorithms.
arXiv:2610.08969v1 Announce Type: new Abstract: Black-box optimization problems are ubiquitous across science and engineering, often dealing with expensive objective functions. This objective latency...
LLINBO is a framework that integrates Large Language Models (LLMs) into Bayesian Optimization (BO) by deciding when to use LLM suggestions and when to fall back on classical BO. It introduces three interaction mechanisms, each with a no‑harm theoretical guarantee of sublinear cumulative regret for common kernels, without requiring performance assumptions on the LLM. Experiments show that LLINBO improves the reliability of LLM‑based BO and outperforms classical BO in both simulations and a real‑world 3D printing task.
arXiv:2608. 03045v1 Announce Type: new Abstract: We consider grey-box optimization problems where the decision variables naturally partition into black-box variables (as arguments to an expensive black-box function) and white-box variables, governed by a set of explicit, closed-form equations that also depend on the output of the black-box function.
arXiv:2603. 07965v2 Announce Type: replace-cross Abstract: Bayesian optimization (BO) for high-dimensional constrained problems remains a significant challenge due to the curse of dimensionality.
arXiv:2610.10186v1 Announce Type: new Abstract: Bayesian optimization (BO) is widely used as a standard approach for expensive black-box optimization. However, BO algorithms often involve parameters...
BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.
arXiv:2607. 29225v1 Announce Type: new Abstract: Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is rarely evaluated alongside optimization performance.
The paper introduces JoBS, a Bayesian optimization framework that co‑optimizes data and model configurations for large language model training without requiring full training runs at each iteration. JoBS first allocates part of the budget to train a scaling‑law‑inspired performance predictor using only a few training steps, then uses this predictor to guide subsequent BO iterations, thereby exploring many more configurations within the same budget. The authors analyze JoBS’s regret, derive optimal budget splits, and demonstrate that it outperforms both independent optimization and existing multi‑fidelity BO baselines across diverse downstream tasks.
arXiv:2609.13396v1 Announce Type: new Abstract: Multi-objective Bayesian optimisation (MOBO) is a sample-efficient approach for optimising expensive black-box functions with multiple objectives. In M...
SABLE is a Bayesian optimisation method designed for high‑dimensional latent spaces where the latent dimension far exceeds the evaluation budget. It uses a Bayesian affine surrogate that decomposes each query into an in‑span component optimized by expected improvement and an off‑span component governed by a scalar norm, with a finite‑horizon schedule to avoid under‑exploration. A recompression scheme reduces the number of pseudo‑observations to ≠O(M), keeping refitting costs manageable while achieving high sample efficiency and rapid decision times on synthetic and latent‑space optimisation tasks.
arXiv:2603. 29730v2 Announce Type: replace-cross Abstract: We present mlr3mbo, a modular toolbox for Bayesian optimization in R.
arXiv:2603. 01470v3 Announce Type: replace Abstract: We consider the optimization problem of an expensive-to-evaluate black-box function, in which we can obtain noisy function values in parallel.