arXiv Machine Learning

ISBO: Scalable Spatio-Temporal Bayesian Optimization with Log Gaussian Cox Process Models via the INLA-SPDE Approach

The paper introduces ISBO, a scalable Spatio‑Temporal Bayesian Optimization framework that models log‑intensity using a Log‑Gaussian Cox Process and performs inference via the INLA‑SPDE approach. By employing a Matern field on meshes, ISBO obtains a sparse Gaussian Markov Random Field, enabling fast and accurate posterior inference during sequential optimization. Experiments on synthetic and real‑world datasets demonstrate accurate peak discovery, intensity recovery, and significant speedups compared to an RKHS‑based baseline.

arXiv Machine Learning
Oct 8

Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling

The paper introduces a scalable Bayesian method for conditional density estimation using a logistic Gaussian process. It models the log conditional density with a separable covariance— a Matérn kernel on the response dimension and a Nyström-feature kernel on covariates— and samples the latent field via kinetic Langevin dynamics instead of traditional approximations. Experiments on photometric‑redshift data with up to 3.9 million observations show the approach matches state‑of‑the‑art tabular foundation models on density and calibration metrics.

By Daniel Paulin, \'Ad\'am Jung, Andr\'as A. Bencz\'ur
arXiv Statistics ML
Oct 2

IQS-BO: In-Context Query Selection for Bayesian Optimisation

IQS-BO introduces an in‑context query selection method for Bayesian optimisation that learns to choose evaluation points via supervised learning on synthetic priors. The approach uses a Prior‑data Fitted Network (PFN) to predict, in a single forward pass, the probability that each candidate maximises the objective, thereby amortising the decision step and eliminating the need for costly surrogate refitting or acquisition maximisation. Experiments show that IQS‑BO matches or surpasses traditional Gaussian‑process‑based BO and other in‑context methods on both synthetic and real‑world benchmarks, and that a mixture prior combining GP samples with challenging function types further improves performance.

By Luca Geminiani, Nadja Klein
arXiv Machine Learning
Jun 9

Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models

arXiv:2606. 08438v1 Announce Type: cross Abstract: Bayesian optimization (BO) is a widely used approach for black-box optimization that uses a Gaussian process (GP) as a surrogate and guides sequential evaluations via an acquisition function, with the ultimate goal of locating the global optimum $\mathbf{x}^{\star}$.

By Yilin Zheng, Haowei Wang, Szu Hui Ng, Enlu Zhou
arXiv Machine Learning
Aug 3

Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery

arXiv:2607. 29225v1 Announce Type: new Abstract: Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is rarely evaluated alongside optimization performance.

By Panagiotis Krokidas, Christoforos Rekatsinas, Vassilis Sioros, Grigorios M. Chatziathanasiou, Efi-Maria Papia, George Giannakopoulos
arXiv Machine Learning
Sep 14

Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization

The paper investigates nonlinear dimensionality reduction for Bayesian optimisation (BO) by transforming high‑dimensional black‑box optimisation problems into a sequence of low‑dimensional latent‑space BO (LSBO) tasks. It extends earlier linear embedding approaches by using variational autoencoders (VAEs), deep metric loss, and adaptive retraining to better capture nonlinear structure, and couples LSBO with sequential domain reduction (SDR‑LSBO) to progressively narrow search domains. Experiments on GPU‑accelerated BoTorch with Matérn‑5/2 Gaussian‑process surrogates show that VAE‑based LSBO outperforms adaptive linear embeddings, and the authors provide a theoretical analysis of latent‑space error versus representation gap under PAC‑Bayes conditions.

By Luo Long, Coralia Cartis, Paz Fink Shustin