The paper investigates how fast predictive regret guarantees of exact Bayesian online learning can be maintained when using approximate posterior methods. It establishes a general theorem linking the cumulative cost of posterior approximation to the contraction radius of the exact Gibbs posterior and the Wasserstein distance between approximate and exact posteriors. Three concrete online learning scenarios—linear models, infinite‑dimensional exponential families, and Gaussian process regression—illustrate that appropriately accurate approximations (projected Langevin, truncation, and sparse variational posteriors) preserve fast regret bounds while reducing computational demands.
By Ilsang Ohn
arXiv:2511. 16340v2 Announce Type: replace Abstract: Efficient Gaussian process (GP) inference is critical for sequential decision-making tasks such as active learning, online prediction, and Bayesian optimization.
By Alan Yufei Dong, Jihao Andreas Lin, Jos\'e Miguel Hern\'andez-Lobato
arXiv:2601.01368v2 Announce Type: replace
Abstract: Score-based causal discovery in the presence of unobserved confounders requires both a consistent scoring criterion and an efficient search over gr...
By Mujin Zhou, Ignavier Ng, Junzhe Zhang
The monograph explores the relationships between Gaussian processes and reproducing kernel Hilbert spaces (RKHS), two widely used approaches that rely on positive definite kernels. It examines how these frameworks connect and are equivalent across key topics such as regression, interpolation, numerical integration, distributional discrepancies, statistical dependence, and Gaussian process sample path properties. By establishing a unifying perspective based on the equivalence between the Gaussian Hilbert space and the RKHS, the work aims to bridge methods developed independently by the machine learning, statistics, and numerical analysis communities.
By Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic, Bharath K. Sriperumbudur
arXiv:2606. 01457v1 Announce Type: new Abstract: Bayesian optimization is a popular way to optimize expensive systems, where every experiment, simulation, or intervention costs time or money.
By Mohammad Ali Javidian
arXiv:2605. 10285v2 Announce Type: replace-cross Abstract: We present a theoretically grounded Gaussian process framework that leverages neural feature maps to construct expressive kernels.
By Anthony Stephenson
arXiv:2608. 20183v1 Announce Type: new Abstract: Classical information criteria such as the Bayesian Information Criterion (BIC) rely on regularity assumptions that break down for singular models, leading to incorrect model selection in settings such as deep learning.
By Gr\'egoire Sergeant-Perthuis (CQSB, Sorbonne Universit\'e), Elias Tsigaridas (Ouragan Team, INRIA), Jules Tsukahara (Ouragan Team, INRIA)
arXiv:2502.16888v3 Announce Type: replace-cross
Abstract: Motivated by the remarkable success of Bayesian additive regression trees (BART) in regression modelling, we propose a novel nonparametric Ba...
By Jiahao Cao, Shiyuan He, Bohai Zhang
arXiv:2606. 18853v1 Announce Type: cross Abstract: A recent line of work has reframed individual decision trees as linear models on engineered features associated with their splits, opening routes for oracle inequalities and feature-importance reinterpretation, but leaving open the question of what unified geometric object a forest induces when one indexes its feature map by nodes rather than by splits.
By Nicolas Mahler
Regression trees are among the most interpretable yet expressive model classes in machine learning. Historically, greedy induction has been the dominant approach for constructing well-performing regression trees.
arXiv:2410. 14483v3 Announce Type: replace-cross Abstract: Reliable uncertainty quantification for causal effects is crucial in high-stakes applications, but remains challenging when the target is an entire function rather than a scalar estimand.
By Hugh Dance, Peter Orbanz, Arthur Gretton
arXiv:2606. 13984v1 Announce Type: cross Abstract: Decision trees are one of the fundamental tools in statistical learning due to their interpretability, flexibility, and their ability to adapt to nonlinear structures.
By Mathias Bourel