arXiv AI
Rethinking Learnability in Offline Data-driven Optimization
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Statistics ML
Oct 2
IQS-BO: In-Context Query Selection for Bayesian Optimisation
IQS-BO introduces an in‑context query selection method for Bayesian optimisation that learns to choose evaluation points via supervised learning on synthetic priors. The approach uses a Prior‑data Fitted Network (PFN) to predict, in a single forward pass, the probability that each candidate maximises the objective, thereby amortising the decision step and eliminating the need for costly surrogate refitting or acquisition maximisation. Experiments show that IQS‑BO matches or surpasses traditional Gaussian‑process‑based BO and other in‑context methods on both synthetic and real‑world benchmarks, and that a mixture prior combining GP samples with challenging function types further improves performance.
arXiv Machine Learning
Oct 7
Direct Regret Optimization in Bayesian Optimization
arXiv:2507.06529v2 Announce Type: replace Abstract: Bayesian optimization (BO) is a powerful paradigm for optimizing expensive black-box functions. Traditional BO methods typically rely on separate h...
arXiv Machine Learning
Oct 15
Tree-Structured Synergy of Large Language Models and Bayesian Optimization for Efficient CASH
The paper introduces LB-MCTS, a trajectory-structured optimization framework that combines Monte Carlo Tree Search, Bayesian Optimization, and Large Language Models to address the CASH problem in AutoML. It uses a shared tree state for algorithm selection, hyperparameter refinement, and synergy between BO and LLM proposals, with a reliability-aware policy that shifts from LLM-driven to BO-driven suggestions as the surrogate model improves. Experiments on 104 AMLB datasets show that LB-MCTS consistently outperforms existing BO-based, LLM-based, and hybrid baselines.
arXiv Machine Learning
Sep 22
D-IMPL: A Diffusion-based Solver for Parameterized BBOs
arXiv:2609. 22752v1 Announce Type: new Abstract: Diffusion models have demonstrated strong power in generative modeling tasks across multiple domains, exhibiting a remarkable capability of learning complex distributions from samples.
arXiv AI
2d ago
Rethinking Bayesian Optimization for Co-Optimizing LLM Training Configurations
The paper introduces JoBS, a Bayesian optimization framework that co‑optimizes data and model configurations for large language model training without requiring full training runs at each iteration. JoBS first allocates part of the budget to train a scaling‑law‑inspired performance predictor using only a few training steps, then uses this predictor to guide subsequent BO iterations, thereby exploring many more configurations within the same budget. The authors analyze JoBS’s regret, derive optimal budget splits, and demonstrate that it outperforms both independent optimization and existing multi‑fidelity BO baselines across diverse downstream tasks.
arXiv Statistics ML
Sep 24
BOCO: Bayesian Online Contextual Optimization for Decision-Focused Online Learning
arXiv:2511.20413v2 Announce Type: replace-cross Abstract: \emph{Decision-focused learning} (DFL) trains predictive models to optimize downstream decisions rather than prediction accuracy alone. While...