← Back to all news
arXiv Machine Learning By Mo Zhou, Weihang Xu, Maryam Fazel, Simon S. Du

Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixtures

Read the original on arXiv Machine Learning →

arXiv:2506. 06584v2 Announce Type: replace Abstract: Learning Gaussian Mixture Models (GMMs) is a fundamental problem in statistics and machine learning, with the Expectation-Maximization (EM) algorithm and its popular variant gradient EM being arguably the most widely used algorithms in practice.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 11

Sublinear Variational Optimization of Gaussian Mixture Models with Millions to Billions of Parameters

The paper introduces a highly efficient variational approximation for Gaussian Mixture Models (GMMs) with arbitrary covariances, integrated with mixtures of factor analyzers. This method reduces the per‑iteration runtime from ≠O(NCD^2) to a complexity that scales linearly with dimensionality D and sublinearly with the product NC. Experiments demonstrate sublinear scaling across the entire optimization, order‑of‑magnitude speed‑ups on large benchmarks, training of GMMs with over 10 billion parameters in under nine hours on a single CPU, and competitive zero‑shot image denoising performance.

By Sebastian Salwig, Till Kahlke, Florian Hirschberger, Dennis Forster, J\"org L\"ucke
benchmarks
More like this →
arXiv Statistics ML
Sep 25

Riemannian Gradient Descent for Gaussian Mixture Models with unknown diagonal covariances

The paper studies the numerical solution of the Beurling‑LASSO (BLASSO) for estimating Gaussian mixture models (GMMs) with unknown numbers of components and unknown diagonal covariance matrices. It introduces a Conic Particle Gradient Descent (CPGD) algorithm that incorporates Riemannian gradient descent to respect the Fisher‑Rao geometry of Gaussian distributions. The authors provide theoretical convergence guarantees, including exponential local convergence under a non‑degeneracy condition related to component separation, and demonstrate through numerical experiments that CPGD is more robust to overspecification of components than the EM algorithm.

By Romane Giard, Yohann De Castro, Roland Denis, Cl\'ement Marteau
efficiency
More like this →
arXiv Machine Learning
Sep 10

The EM-algorithm and the Method of Moments in Softmax Mixture Models

arXiv:2409. 09903v3 Announce Type: replace-cross Abstract: Softmax Mixture Models (SMMs) are discrete $K$-component mixture models for the probabilities of selecting one of $p$ candidate feature vectors $X_1,\ldots,X_p\in\mathbb{R}^L$ in heterogeneous populations and are widely used in econometrics and scientific applications.

By Xin Bing, Florentina Bunea, Jonathan Niles-Weed, Marten Wegkamp
llms
More like this →