arXiv Statistics ML

Online Covariance Estimation for Averaged Stochastic Gradient Descent on Riemannian Manifolds

arXiv Statistics ML
Sep 25

Riemannian Gradient Descent for Gaussian Mixture Models with unknown diagonal covariances

The paper studies the numerical solution of the Beurling‑LASSO (BLASSO) for estimating Gaussian mixture models (GMMs) with unknown numbers of components and unknown diagonal covariance matrices. It introduces a Conic Particle Gradient Descent (CPGD) algorithm that incorporates Riemannian gradient descent to respect the Fisher‑Rao geometry of Gaussian distributions. The authors provide theoretical convergence guarantees, including exponential local convergence under a non‑degeneracy condition related to component separation, and demonstrate through numerical experiments that CPGD is more robust to overspecification of components than the EM algorithm.

By Romane Giard, Yohann De Castro, Roland Denis, Cl\'ement Marteau
arXiv Statistics ML
Sep 4

Online Learning of Functional Principal Component Analysis for Multidimensional Functional Data

The paper introduces an online framework for functional principal component analysis (FPCA) tailored to multidimensional functional data streams. It models functional principal components with tensor product splines, enforcing smoothness and orthonormality via a penalized approach on a Stiefel manifold. The authors present efficient Riemannian stochastic gradient descent and AdaGrad algorithms, along with a dynamic smoothing parameter tuning strategy based on rolling block validation, and provide asymptotic normality results and confidence intervals for the estimators.

By Muye Nanshan, Nan Zhang, Jiguo Cao
arXiv Machine Learning
Sep 21

Riemannian Simultaneous Inference for Tangent Vector Field Regression

The paper develops a nonparametric kernel estimator for tangent vector field regression on a Riemannian manifold without boundary, using parallel transport to align responses before averaging. It derives uniform bias, covariance, and stochastic rates, and constructs a simultaneous confidence tube by whitening the tangent norm into a unit‑variance Gaussian field, whose Gumbel limit yields an explicit intrinsic constant. The method is validated through simulations on various manifolds and applied to reconstruct global wind data, demonstrating spatially varying uncertainty.

By Xiaotian Chang, Yangdi Jiang, Qirui Hu
arXiv Machine Learning
Oct 5

Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models

The paper introduces Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models, enabling the variational posterior over latent variables to depend on GP inducing points. This approach addresses limitations of mean‑field approximations and yields improved reconstruction metrics for data manifold points.

By Maksym Tretiakov, Sarah Lucie Filipp, Vincent Fortuin, Ruth Misener, Ruby Sedgwick, James Odgers
arXiv Machine Learning
Jun 18

Riemannian MeanFlow for One-Step Generation on Manifolds

arXiv:2603. 10718v3 Announce Type: replace Abstract: Flow Matching enables simulation-free training of generative models on Riemannian manifolds, yet sampling typically still relies on numerically integrating a probability-flow ODE.

By Zichen Zhong, Haoliang Sun, Yukun Zhao, Yongshun Gong, Yilong Yin
arXiv Machine Learning
Jul 7

A Gradient Flow Perspective on Minimum MMD Estimation

arXiv:2607. 03871v1 Announce Type: new Abstract: Minimum maximum mean discrepancy (MMD) estimation has emerged as a robust and likelihood-free alternative to maximum likelihood estimation for parameter estimation.

By Sophia Seulkee Kang, Louis Sharrock, Xiaoyuan Cheng, Fran\c{c}ois-Xavier Briol, Zonghao Chen
arXiv Machine Learning
Aug 20

Inference and Uncertainty Quantification for Streaming $r$-PCA

The paper tackles two key gaps in streaming PCA using Oja's algorithm: it establishes sharp operator‑norm convergence for general‑rank subspaces under sub‑Gaussian data, and it provides distributional inference for the resulting subspace estimator. The authors remove non‑vanishing remainder terms from existing analyses, achieving rates that match minimax bounds in both dense‑tail and sparse‑tail regimes. They further develop a linearization of Oja’s iterates, enabling high‑dimensional Gaussian approximations and an online multiplier bootstrap for practical inference.

By Haoshu Xu, Hongzhe Li