The paper argues against using the Gaussian (squared exponential/RBF) kernel as a default in Gaussian process regression, citing its brittleness. Two key issues are highlighted: the kernel produces unrealistically small conditional variances leading to overconfident predictions, and it causes numerical ill‑conditioning that necessitates ad‑hoc fixes like nugget terms. The authors attribute these problems to the kernel’s analytic, infinitely smooth nature, suggesting that analytic stationary kernels in general should be avoided.
By Toni Karvonen, Chris J. Oates
The monograph explores the relationships between Gaussian processes and reproducing kernel Hilbert spaces (RKHS), two widely used approaches that rely on positive definite kernels. It examines how these frameworks connect and are equivalent across key topics such as regression, interpolation, numerical integration, distributional discrepancies, statistical dependence, and Gaussian process sample path properties. By establishing a unifying perspective based on the equivalence between the Gaussian Hilbert space and the RKHS, the work aims to bridge methods developed independently by the machine learning, statistics, and numerical analysis communities.
By Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic, Bharath K. Sriperumbudur
arXiv:2510. 25599v2 Announce Type: replace Abstract: Regression tasks, notably in safety-critical domains, require reliable uncertainty quantification, yet the literature remains largely classification-focused.
By Christopher B\"ulte, Yusuf Sale, Gitta Kutyniok, Eyke H\"ullermeier
arXiv:2512. 06143v2 Announce Type: replace Abstract: Despite a large corpus of recent work on scaling up Gaussian processes, a stubborn trade-off between computational speed, prediction and uncertainty quantification accuracy, and customizability persists.
By Marcus M. Noack, Mark D. Risser, Hengrui Luo, Vardaan Tekriwal, Ronald J. Pandolfi
arXiv:2603. 16481v3 Announce Type: replace Abstract: Non-conservative uncertainty bounds are essential for making reliable predictions about latent functions from noisy data, and thus, a key enabler for safe learning-based control.
By Amon Lahr, Anna Scampicchio, Johannes K\"ohler, Melanie N. Zeilinger
arXiv:2602. 23006v2 Announce Type: replace-cross Abstract: Simulating a Gaussian process requires sampling from a high-dimensional Gaussian distribution, which scales cubically with the number of sample locations.
By Arsalan Jawaid, Abdullah Karatas, J\"org Seewig
arXiv:2605. 10285v2 Announce Type: replace-cross Abstract: We present a theoretically grounded Gaussian process framework that leverages neural feature maps to construct expressive kernels.
By Anthony Stephenson
arXiv:2607. 21823v1 Announce Type: new Abstract: We show that, up to isotropic scaling, the Gaussian RBF reproducing kernel Hilbert space (RKHS) is asymptotically isometric to Euclidean space in the large bandwidth limit.
By Sergio A. Alvarez
arXiv:2604. 03146v2 Announce Type: replace-cross Abstract: We study high-dimensional convex empirical risk minimization (ERM) under general non-Gaussian data designs.
By Chiheb Yaakoubi, Cosme Louart, Malik Tiomoko, Zhenyu Liao
arXiv:2401. 01599v4 Announce Type: replace Abstract: The generalization error curve of certain kernel regression method aims at determining the exact order of generalization error with various source condition, noise level and choice of the regularization parameter rather than the minimax rate.
By Yicheng Li, Weiye Gan, Zuoqiang Shi, Qian Lin
arXiv:2606. 07561v1 Announce Type: new Abstract: Gaussian processes with stationary kernels on bounded domains exhibit inflated posterior variance near the boundary.
By Maria B{\aa}nkestad, Sanna Jarl, Jens Sj\"olund
arXiv:2606. 01427v1 Announce Type: cross Abstract: Foundation models (FMs) have achieved substantial success in generalizing across tasks without problemspecific training or fine-tuning.
By Tyler R. Johnson, Kian Ben-Jacob, Nima Negarandeh, Oriol Vendrell-Gallart, Ramin Bostanabad