The paper argues against using the Gaussian (squared exponential/RBF) kernel as a default in Gaussian process regression, citing its brittleness. It shows that the kernel leads to unrealistically small conditional variances, causing overconfidence in predictive uncertainty, and that this small variance induces numerical ill‑conditioning, necessitating tricks like nugget terms that alter the model. The authors attribute these issues to the kernel’s analytic, highly smooth nature and suggest that analytic stationary kernels in general should be avoided.
arXiv:2610.09731v1 Announce Type: cross
Abstract: Kernel methods are among the most powerful tools in machine learning and statistics, with a large number of successful applications. Their immense su...
By Jose Cribeiro-Ramallo, Florian Kalinke, Zolt\'an Szab\'o
The paper explores the relationship between Gaussian processes and Gaussian random elements within reproducing kernel Banach spaces. It demonstrates that the covariance operator of a weak second‑order Radon probability measure on such a space is uniquely defined by a symmetric positive‑definite function, and in the Gaussian case it identifies which functions correspond to covariance operators via γ‑radonifying operators. Leveraging these findings, the authors extend Driscoll’s theorem to the setting of reproducing kernel Banach spaces.
By Toni Karvonen, Rasmus Kleist H{\o}rlyck S{\o}rensen
arXiv:2403.12187v2 Announce Type: replace-cross
Abstract: Motivated by the abundance of functional data, such as time series and images, we study the approximation and statistical learning of nonline...
By Tian-Yi Zhou, Namjoon Suh, Guang Cheng, Xiaoming Huo
The paper argues against using the Gaussian (squared exponential/RBF) kernel as a default in Gaussian process regression, citing its brittleness. Two key issues are highlighted: the kernel produces unrealistically small conditional variances leading to overconfident predictions, and it causes numerical ill‑conditioning that necessitates ad‑hoc fixes like nugget terms. The authors attribute these problems to the kernel’s analytic, infinitely smooth nature, suggesting that analytic stationary kernels in general should be avoided.
By Toni Karvonen, Chris J. Oates
arXiv:2610.33286v1 Announce Type: cross
Abstract: Given a random probability distribution \(F\), whose realizations are defined on \(\mathbb R^d\), and a kernel \(k\), our aim is to analyze the prope...
By Amparo Ba\'illo, Mart\'in S\'anchez-Signorini
arXiv:2603. 16481v3 Announce Type: replace Abstract: Non-conservative uncertainty bounds are essential for making reliable predictions about latent functions from noisy data, and thus, a key enabler for safe learning-based control.
By Amon Lahr, Anna Scampicchio, Johannes K\"ohler, Melanie N. Zeilinger
arXiv:2608.28446v1 Announce Type: cross
Abstract: For finite-dimensional linear inverse problems where the variables are Gaussian, it is well-known that the minimum-mean-square error estimator takes...
By Michael Unser
arXiv:2602. 23006v2 Announce Type: replace-cross Abstract: Simulating a Gaussian process requires sampling from a high-dimensional Gaussian distribution, which scales cubically with the number of sample locations.
By Arsalan Jawaid, Abdullah Karatas, J\"org Seewig
We develop a comprehensive theory for regularized M-estimation in reproducing kernel Hilbert spaces. Under mild conditions on the loss we establish existence and measurability of the estimator, covering a wide range of convex and non-convex losses, including bounded robust losses.
The paper introduces a unified framework for measuring divergences between conditional distributions using kernel embeddings, termed conditional maximum mean discrepancy (CMMD). It defines a family of CMMD metrics—CMMD_0, CMMD_1, and CMMD_2—each based on different RKHS embeddings, and generalizes them to a level‑s CMMD with clear assumptions and operator‑based smoothing connections. Additionally, the authors present a novel doubly robust estimator for CMMD and demonstrate through experiments that the metric captures complex conditional dependencies for statistical testing.
By Peter Moskvichev, Siu Lun Chau, Dino Sejdinovic
arXiv:2610.10080v1 Announce Type: cross
Abstract: Gaussian processes (GPs) are widely used across machine learning, spatial statistics, time-series analysis, optimization, Bayesian statistics, and sc...
By Ameer Qaqish, Didong Li