arXiv Machine Learning

Towards Regret Guarantees for One-Step Lookahead Bayesian Optimization

The paper investigates theoretical guarantees for a one‑step lookahead Bayesian optimization method called optimal‑point variance reduction (OVR). It derives a uniform Monte Carlo estimation error bound for the acquisition function and shows that a regularized version of OVR achieves a vanishing Bayesian expected simple regret bound. Numerical experiments validate the performance of both OVR and its regularized variant.

arXiv Machine Learning
Jun 8

$\alpha$-PFN: Fast Entropy Search via In-Context Learning

arXiv:2606. 07134v1 Announce Type: new Abstract: Information-theoretic acquisition functions such as Entropy Search (ES) offer a principled exploration-exploitation framework for Bayesian optimization (BO).

By Herilalaina Rakotoarison, Steven Adriaensen, Tom Viering, Carl Hvarfner, Samuel M\"uller, Frank Hutter, Eytan Bakshy
arXiv Machine Learning
Jun 2

Local Preferential Bayesian Optimization

arXiv:2606. 02351v1 Announce Type: new Abstract: Bayesian optimization (BO) is a popular and effective approach for tuning expensive, noisy experiments, but requires the formulation of an explicit objective function.

By Johanna Menn, Miriam Kober, Paul Brunzema, David Stenger, Sebastian Trimpe
arXiv Machine Learning
Aug 27

Fast rates in Bayesian online learning with approximate posteriors

The paper investigates how fast predictive regret guarantees of exact Bayesian online learning can be maintained when using approximate posterior methods. It establishes a general theorem linking the cumulative cost of posterior approximation to the contraction radius of the exact Gibbs posterior and the Wasserstein distance between approximate and exact posteriors. Three concrete online learning scenarios—linear models, infinite‑dimensional exponential families, and Gaussian process regression—illustrate that appropriately accurate approximations (projected Langevin, truncation, and sparse variational posteriors) preserve fast regret bounds while reducing computational demands.

By Ilsang Ohn