← Back to all news
arXiv Machine Learning

Online Adaptive Kernel Mixing for Gaussian Process Decision Making

The paper introduces HACK GPs, a method that treats kernel selection for Gaussian Processes as an online learning problem with expert advice. Each candidate kernel is viewed as a GP expert, and a distribution over these experts is updated online using AdaHedge based on a loss that reflects both function fit and task alignment. Two variants—Mixture of Gaussians and categorical sampling—are presented, with theoretical guarantees that the weight concentrates on the best kernel under a loss‑gap condition, and empirical results show robust performance across Bayesian optimization, level set estimation, and Bayesian active learning compared to standard kernels and simple ensembles.

Related stories

arXiv AI
Aug 26

Enhancing Bayesian Optimization and Active Learning Through Kernel Diversity

The paper introduces KENDO, a unified framework that combines Ensemble Gaussian Processes with disagreement‑aware acquisition strategies to address hyperparameter selection in Bayesian optimization and active learning. By replacing costly hyperparameter sampling with a kernel ensemble and adaptive Bayesian weighting, KENDO‑BO and KENDO‑AL provide self‑correcting mechanisms tailored to their respective tasks. Experiments on synthetic and real‑world benchmarks show that KENDO‑BO matches or outperforms state‑of‑the‑art methods while cutting computational cost up to fivefold, and KENDO‑AL delivers better predictive calibration with up to 27‑times speedup compared to MCMC‑based baselines.

By Heng Zhang, Haotian Xiang, Qin Lu, Konstantinos D. Polyzos, Tara Javidi
benchmarks
More like this →
arXiv Machine Learning
Aug 24

Amortized Bandwidth Learning for Kernel Density Estimation under Logarithmic Score

The paper introduces an amortized learning framework for selecting bandwidths in kernel density estimation by optimizing the logarithmic score across a distribution of tasks. It uses a truncated-and-renormalized bounded-support formulation and affine standardization to achieve stable learning and transferability across different intervals. Experiments on Gaussian samples, a multi-family benchmark, and randomized Gaussian mixtures demonstrate that the learned selector outperforms traditional methods such as Silverman’s rule, Sheather–Jones, and least‑squares cross‑validation, especially for small or heterogeneous samples.

By Junyi Liang, Hailiang Du
benchmarks
More like this →
arXiv Machine Learning
Jul 23

Adaptive Bayesian Online Learning via Expert Aggregation

arXiv:2607. 20239v1 Announce Type: cross Abstract: Bayesian online learning promises uncertainty-aware prediction on data streams, but its performance hinges on inferential choices, including learning rates, prior distributions and variational families, which are usually fixed before seeing the stream.

By Jungbin Jun, Ilsang Ohn
More like this →
arXiv Machine Learning
Aug 27

Gradient-based Sample Selection for Faster Bayesian Optimization

The paper introduces Gradient-based Sample Selection Bayesian Optimization (GSSBO), a method that builds the Gaussian process surrogate on a strategically chosen subset of samples rather than the full dataset. By using gradient information to eliminate redundant points while keeping diversity and representativeness, GSSBO achieves sublinear regret bounds and reduces the cubic computational cost of standard BO. Experiments on synthetic and real-world tasks show that this approach maintains comparable optimization performance while significantly cutting GP fitting time and resource usage.

By Qiyu Wei, Haowei Wang, Zirui Cao, Songhao Wang, Richard Allmendinger, Mauricio A \'Alvarez
More like this →