← Back to all news
arXiv Machine Learning By Chih-Yu Chang, Milad Azvar, Chinedum Okwudire, Raed Al Kontar

LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization

Read the original on arXiv Machine Learning →

LLINBO is a framework that integrates Large Language Models (LLMs) into Bayesian Optimization (BO) by deciding when to use LLM suggestions and when to fall back on classical BO. It introduces three interaction mechanisms, each with a no‑harm theoretical guarantee of sublinear cumulative regret for common kernels, without requiring performance assumptions on the LLM. Experiments show that LLINBO improves the reliability of LLM‑based BO and outperforms classical BO in both simulations and a real‑world 3D printing task.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
6d ago

Rethinking Bayesian Optimization for Co-Optimizing LLM Training Configurations

The paper introduces JoBS, a Bayesian optimization framework that co‑optimizes data and model configurations for large language model training without requiring full training runs at each iteration. JoBS first allocates part of the budget to train a scaling‑law‑inspired performance predictor using only a few training steps, then uses this predictor to guide subsequent BO iterations, thereby exploring many more configurations within the same budget. The authors analyze JoBS’s regret, derive optimal budget splits, and demonstrate that it outperforms both independent optimization and existing multi‑fidelity BO baselines across diverse downstream tasks.

By Zhiliang Chen, Alfred Wei Lun Leong, Shao Yong Ong, Apivich Hemachandra, Gregory Kang Ruey Lau, Chuan-Sheng Foo, Zhengyuan Liu, Nancy F. Chen, Bryan Kian Hsiang Low
llmsfine-tuning
More like this →
arXiv AI
Oct 28

BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.

By Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra, Bryan Kian Hsiang Low
llmsbenchmarks
More like this →