The paper argues against using the Gaussian (squared exponential/RBF) kernel as a default in Gaussian process regression, citing its brittleness. It shows that the kernel leads to unrealistically small conditional variances, causing overconfidence in predictive uncertainty, and that this small variance induces numerical ill‑conditioning, necessitating tricks like nugget terms that alter the model. The authors attribute these issues to the kernel’s analytic, highly smooth nature and suggest that analytic stationary kernels in general should be avoided.
arXiv:2610.09731v1 Announce Type: cross
Abstract: Kernel methods are among the most powerful tools in machine learning and statistics, with a large number of successful applications. Their immense su...
By Jose Cribeiro-Ramallo, Florian Kalinke, Zolt\'an Szab\'o
The paper explores the relationship between Gaussian processes and Gaussian random elements within reproducing kernel Banach spaces. It demonstrates that the covariance operator of a weak second‑order Radon probability measure on such a space is uniquely defined by a symmetric positive‑definite function, and in the Gaussian case it identifies which functions correspond to covariance operators via γ‑radonifying operators. Leveraging these findings, the authors extend Driscoll’s theorem to the setting of reproducing kernel Banach spaces.
By Toni Karvonen, Rasmus Kleist H{\o}rlyck S{\o}rensen
arXiv:2403.12187v2 Announce Type: replace-cross
Abstract: Motivated by the abundance of functional data, such as time series and images, we study the approximation and statistical learning of nonline...
By Tian-Yi Zhou, Namjoon Suh, Guang Cheng, Xiaoming Huo
The paper argues against using the Gaussian (squared exponential/RBF) kernel as a default in Gaussian process regression, citing its brittleness. Two key issues are highlighted: the kernel produces unrealistically small conditional variances leading to overconfident predictions, and it causes numerical ill‑conditioning that necessitates ad‑hoc fixes like nugget terms. The authors attribute these problems to the kernel’s analytic, infinitely smooth nature, suggesting that analytic stationary kernels in general should be avoided.
By Toni Karvonen, Chris J. Oates
arXiv:2610.33286v1 Announce Type: cross
Abstract: Given a random probability distribution \(F\), whose realizations are defined on \(\mathbb R^d\), and a kernel \(k\), our aim is to analyze the prope...
By Amparo Ba\'illo, Mart\'in S\'anchez-Signorini