In-Context Batch Black-box Optimization via Flow Matching introduces FMTR, an amortized method that learns a continuous, history-conditioned proposal distribution for variable-size batch sampling in expensive black-box optimization settings. FMTR is trained offline using conditional flow matching on Thompson-sampling argmaxes refined through a horizon-dependent Langevin pushforward map, enabling adaptive exploration-exploitation over the optimization horizon. At test time, FMTR generates batches without task-specific surrogate refitting or numerical acquisition optimization, achieving strong performance on synthetic and real-world benchmarks while significantly reducing batch selection computational cost.
By Quoc-Anh Hoang Nguyen, Xinyu Zhang, Daolang Huang, Julien Martinelli, Samuel Kaski
arXiv:2406.04261v2 Announce Type: replace
Abstract: Simulators are vital in science and engineering, as they faithfully model the influence of design parameters on real-world observations. Optimizing...
By Tim Bakker, Fabio Valerio Massoli, Thomas Hehn, Tribhuvanesh Orekondy, Arash Behboodi
The paper introduces LB-MCTS, a trajectory-structured optimization framework that combines Monte Carlo Tree Search, Bayesian Optimization, and Large Language Models to address the CASH problem in AutoML. It uses a shared tree state for algorithm selection, hyperparameter refinement, and synergy between BO and LLM proposals, with a reliability-aware policy that shifts from LLM-driven to BO-driven suggestions as the surrogate model improves. Experiments on 104 AMLB datasets show that LB-MCTS consistently outperforms existing BO-based, LLM-based, and hybrid baselines.
By Beicheng Xu, Weitong Qian, Lingching Tung, Yupeng Lu, Bin Cui
arXiv:2607. 00691v1 Announce Type: new Abstract: Black-box optimization is a fundamental science and engineering tool that makes it possible to optimize objectives without gradient information.
By Edouard R. Dufour, Pascal Fua
The paper introduces Gradient-based Sample Selection Bayesian Optimization (GSSBO), a method that builds the Gaussian process surrogate on a strategically chosen subset of samples rather than the full dataset. By using gradient information to eliminate redundant points while keeping diversity and representativeness, GSSBO achieves sublinear regret bounds and reduces the cubic computational cost of standard BO. Experiments on synthetic and real-world tasks show that this approach maintains comparable optimization performance while significantly cutting GP fitting time and resource usage.
By Qiyu Wei, Haowei Wang, Zirui Cao, Songhao Wang, Richard Allmendinger, Mauricio A \'Alvarez
GRAPE is a two‑stage Bayesian optimization framework that first refines the local gradient posterior using a closed‑form acquisition function and then selects update directions by maximizing expected decrease conditioned on descent. The authors prove that the refinement stage monotonically reduces local uncertainty and that the progress‑aware direction converges to true steepest descent as the posterior sharpens. Empirical results show GRAPE achieves a 5.4× speedup on black‑box adversarial attacks and reduces final average regret by 3.8 log‑units on large language model prompt‑optimization tasks.
By Richard Cornelius Suwandi, Feng Yin
The paper investigates training diffusion models to sample from distributions defined by unnormalized densities or energy functions. It benchmarks various diffusion-structured inference techniques, including simulation-based variational methods and off-policy approaches such as continuous generative flow networks, highlighting their relative strengths and challenging some prior claims. Additionally, the authors introduce a new exploration strategy for off-policy methods that employs local search in the target space with a replay buffer, demonstrating improved sample quality across multiple target distributions.
By Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca, Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, Esmeralda S. Whitammer
arXiv:2508. 12116v2 Announce Type: replace-cross Abstract: As numerous instruction-tuning datasets continue to emerge, dynamically balancing and optimizing their mixtures has become a critical challenge.
By Haebin Shin, Lei Ji, Xiao Liu, Zhiwei Yu, Hyunwoo Yoo, Qi Chen, Yeyun Gong
arXiv:2610.16984v1 Announce Type: cross
Abstract: Offline reinforcement learning aims to learn effective policies from fixed datasets without environment interaction. Although generative models such...
By Shenghua Lu, Zixuan Yang, Rongxu Zhao, Bohao Qu, Jifeng Hu, Hechang Chen
arXiv:2610.10186v1 Announce Type: new
Abstract: Bayesian optimization (BO) is widely used as a standard approach for expensive black-box optimization. However, BO algorithms often involve parameters...
By Satoshi Katayama, Shoyo Hunt, Shintaro Masuda, Masayuki Karasuyama
arXiv:2609.01493v1 Announce Type: cross
Abstract: Black-Box Optimization (BBO) has found broad applications, but evolutionary algorithms and Bayesian optimization face efficiency challenges as real-w...
By Chao Qian, Chen-Guang Wang, Rong-Xi Tan, Ke Xue
arXiv:2606. 30228v1 Announce Type: new Abstract: Modern engineering workflows increasingly rely on massive parallel simulation, driving the need for scalable, large-batch Bayesian Optimization (BO).
By Maximilian Bloor, Liyuan Xu, Hrvoje Stojic, Victor Picheny