The paper introduces HACK GPs, a method that treats kernel selection for Gaussian Processes as an online learning problem with expert advice. Each candidate kernel is viewed as a GP expert, and a distribution over these experts is updated online using AdaHedge based on a loss that reflects both function fit and task alignment. Two variants—Mixture of Gaussians and categorical sampling—are presented, with theoretical guarantees that the weight concentrates on the best kernel under a loss‑gap condition, and empirical results show robust performance across Bayesian optimization, level set estimation, and Bayesian active learning compared to standard kernels and simple ensembles.
By Kavin Aravindan, Mani Tej Sriram, Gautam Dasarathy, Tejas Bodas
arXiv:2507.06529v2 Announce Type: replace
Abstract: Bayesian optimization (BO) is a powerful paradigm for optimizing expensive black-box functions. Traditional BO methods typically rely on separate h...
By Fengxue Zhang, Yuxin Chen
arXiv:2601. 07094v2 Announce Type: replace-cross Abstract: Bayesian optimization (BO) iteratively fits a Gaussian process (GP) surrogate to accumulated evaluations and selects new queries via an acquisition function.
By Jiguang Li, Hengrui Luo
HyperMC is a multi‑fidelity hyperparameter tuning framework for stochastic gradient Markov chain Monte Carlo (SGMCMC) that combines Hyperband-style resource allocation with kernel Stein discrepancy (KSD) evaluation. It uses successive‑halving brackets to explore a continuous hyperparameter space while progressively refining promising configurations within a fixed computational budget. Robust HyperMC further introduces global grid initialization and elite‑guided local refinement to reduce sensitivity to random candidate generation and noisy evaluations, and theoretical analysis shows that the successive‑halving component selects a near‑optimal configuration with high probability under suitable conditions.
By Ming Tan, Xiyun Jiao
The paper explores how to use gradients of simulator outputs with respect to input parameters to improve Bayesian inference for expensive simulators. By incorporating gradient information into Gaussian process surrogate models, the authors show that Bayesian optimization can converge faster under a limited simulation budget. They find that reverse‑mode differentiation yields efficiency gains, while forward‑mode does not offset its computational cost, especially when the number of parameters exceeds the output dimensionality.
By \v{S}imon Sold\'at, V\'aclav \v{S}m\'idl
arXiv:2510.09877v4 Announce Type: replace-cross
Abstract: Over the past couple of decades, many active learning acquisition functions have been proposed, leaving practitioners with an unclear choice...
By Kangping Hu, Stephen Mussmann
arXiv:2610.08969v1 Announce Type: new
Abstract: Black-box optimization problems are ubiquitous across science and engineering, often dealing with expensive objective functions. This objective latency...
By Gustavo Sutter, Alejandro Comas-Leon, David Holzm\"uller, Hao Wang, Luis Ricardez-Sandoval, Pascal Poupart, Agustinus Kristiadi
The paper introduces Gradient-based Sample Selection Bayesian Optimization (GSSBO), a method that builds the Gaussian process surrogate on a strategically chosen subset of samples rather than the full dataset. By using gradient information to eliminate redundant points while keeping diversity and representativeness, GSSBO achieves sublinear regret bounds and reduces the cubic computational cost of standard BO. Experiments on synthetic and real-world tasks show that this approach maintains comparable optimization performance while significantly cutting GP fitting time and resource usage.
By Qiyu Wei, Haowei Wang, Zirui Cao, Songhao Wang, Richard Allmendinger, Mauricio A \'Alvarez
The paper introduces an amortized learning framework for selecting bandwidths in kernel density estimation by optimizing the logarithmic score across a distribution of tasks. It uses a truncated-and-renormalized bounded-support formulation and affine standardization to achieve stable learning and transferability across different intervals. Experiments on Gaussian samples, a multi-family benchmark, and randomized Gaussian mixtures demonstrate that the learned selector outperforms traditional methods such as Silverman’s rule, Sheather–Jones, and least‑squares cross‑validation, especially for small or heterogeneous samples.
By Junyi Liang, Hailiang Du
arXiv:2511. 02570v3 Announce Type: replace Abstract: Bayesian optimization (BO) is a widely used approach to hyperparameter optimization (HPO).
By Lukas Fehring, Marcel Wever, Maximilian Splieth\"over, Leona Hennig, Henning Wachsmuth, Marius Lindauer
arXiv:2608. 00641v2 Announce Type: replace Abstract: Bayesian optimization (BO) relies on a surrogate model and an acquisition function, yet the most suitable choices vary across tasks and optimization stages.
By Changquan Zhao, Yuxiang Sun, Ruihao Zhu, Cheng Hua, Yulian He
IQS-BO introduces an in‑context query selection method for Bayesian optimisation that learns to choose evaluation points via supervised learning on synthetic priors. The approach uses a Prior‑data Fitted Network (PFN) to predict, in a single forward pass, the probability that each candidate maximises the objective, thereby amortising the decision step and eliminating the need for costly surrogate refitting or acquisition maximisation. Experiments show that IQS‑BO matches or surpasses traditional Gaussian‑process‑based BO and other in‑context methods on both synthetic and real‑world benchmarks, and that a mixture prior combining GP samples with challenging function types further improves performance.
By Luca Geminiani, Nadja Klein