IQS-BO introduces an in‑context query selection method for Bayesian optimisation that learns to choose evaluation points via supervised learning on synthetic priors. The approach uses a Prior‑data Fitted Network (PFN) to predict, in a single forward pass, the probability that each candidate maximises the objective, thereby amortising the decision step and eliminating the need for costly surrogate refitting or acquisition maximisation. Experiments show that IQS‑BO matches or surpasses traditional Gaussian‑process‑based BO and other in‑context methods on both synthetic and real‑world benchmarks, and that a mixture prior combining GP samples with challenging function types further improves performance.
By Luca Geminiani, Nadja Klein
arXiv:2507.06529v2 Announce Type: replace
Abstract: Bayesian optimization (BO) is a powerful paradigm for optimizing expensive black-box functions. Traditional BO methods typically rely on separate h...
By Fengxue Zhang, Yuxin Chen
The paper introduces LB-MCTS, a trajectory-structured optimization framework that combines Monte Carlo Tree Search, Bayesian Optimization, and Large Language Models to address the CASH problem in AutoML. It uses a shared tree state for algorithm selection, hyperparameter refinement, and synergy between BO and LLM proposals, with a reliability-aware policy that shifts from LLM-driven to BO-driven suggestions as the surrogate model improves. Experiments on 104 AMLB datasets show that LB-MCTS consistently outperforms existing BO-based, LLM-based, and hybrid baselines.
By Beicheng Xu, Weitong Qian, Lingching Tung, Yupeng Lu, Bin Cui
arXiv:2609. 22752v1 Announce Type: new Abstract: Diffusion models have demonstrated strong power in generative modeling tasks across multiple domains, exhibiting a remarkable capability of learning complex distributions from samples.
By Yang Hu, Na Li
The paper introduces JoBS, a Bayesian optimization framework that co‑optimizes data and model configurations for large language model training without requiring full training runs at each iteration. JoBS first allocates part of the budget to train a scaling‑law‑inspired performance predictor using only a few training steps, then uses this predictor to guide subsequent BO iterations, thereby exploring many more configurations within the same budget. The authors analyze JoBS’s regret, derive optimal budget splits, and demonstrate that it outperforms both independent optimization and existing multi‑fidelity BO baselines across diverse downstream tasks.
By Zhiliang Chen, Alfred Wei Lun Leong, Shao Yong Ong, Apivich Hemachandra, Gregory Kang Ruey Lau, Chuan-Sheng Foo, Zhengyuan Liu, Nancy F. Chen, Bryan Kian Hsiang Low
arXiv:2511.20413v2 Announce Type: replace-cross
Abstract: \emph{Decision-focused learning} (DFL) trains predictive models to optimize downstream decisions rather than prediction accuracy alone. While...
By Zhuojun Xie, Adam Abdin, Yiping Fang
arXiv:2606. 00366v1 Announce Type: new Abstract: We consider the problem of generating a large collection of initial guesses for local minima of multimodal non-convex continuous optimization problems.
By Anjian Li, Bartolomeo Stellato, Ryne Beeson
BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.
By Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra, Bryan Kian Hsiang Low
arXiv:2606. 00862v1 Announce Type: cross Abstract: Surrogate-assisted evolutionary algorithms (SAEAs) have been widely used for expensive black-box optimization problems.
By Xiao Jin, Yongxiong Wang, Haobo Liu, Yudong Du, Yukun Du
arXiv:2606. 07841v1 Announce Type: cross Abstract: Black-box variational inference (BBVI) is a methodology for posterior approximation that relies on stochastic optimization.
By Trevor Campbell, Jonathan H. Huggins, Kyurae Kim, Charles C. Margossian
arXiv:2406.04261v2 Announce Type: replace
Abstract: Simulators are vital in science and engineering, as they faithfully model the influence of design parameters on real-world observations. Optimizing...
By Tim Bakker, Fabio Valerio Massoli, Thomas Hehn, Tribhuvanesh Orekondy, Arash Behboodi
Offline multi-objective optimization (MOO) seeks solutions with better objective trade-offs using only a fixed dataset, without querying the objectives. Diffusion models trained on such data have emer...