arXiv Statistics ML

Differentiable Expectation-Maximisation and Applications to Gaussian Mixture Model Optimal Transport

arXiv Machine Learning
Jun 5

Variational Entropic Optimal Transport

arXiv:2602. 02241v2 Announce Type: replace Abstract: Entropic optimal transport (EOT) in continuous spaces with quadratic cost is a classical tool for solving the domain translation problem.

By Roman Dyachenko, Nikita Gushchin, Kirill Sokolov, Petr Mokrov, Evgeny Burnaev, Alexander Korotin
arXiv Machine Learning
Aug 19

Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixtures

arXiv:2506. 06584v2 Announce Type: replace Abstract: Learning Gaussian Mixture Models (GMMs) is a fundamental problem in statistics and machine learning, with the Expectation-Maximization (EM) algorithm and its popular variant gradient EM being arguably the most widely used algorithms in practice.

By Mo Zhou, Weihang Xu, Maryam Fazel, Simon S. Du
arXiv AI
Jun 16

Optimal Transport for Machine Learners

arXiv:2505. 06589v2 Announce Type: replace-cross Abstract: Modern machine learning repeatedly manipulates probability measures: empirical datasets, generated samples, latent distributions, class-conditional laws, particle systems, weights of wide networks and attention patterns.

By Gabriel Peyr\'e
arXiv Machine Learning
Aug 6

E$^2$M: Double Bounded $\alpha$-Divergence Optimization for Tensor-based Discrete Density Estimation

arXiv:2405. 18220v4 Announce Type: replace-cross Abstract: Tensor-based discrete density estimation requires flexible modeling and proper divergence criteria to enable effective learning; however, traditional approaches using $\alpha$-divergence face analytical challenges due to the $\alpha$-power terms in the objective function, which hinder the derivation of closed-form update rules.

By Kazu Ghalamkari, Jesper L{\o}ve Hinrich, Morten M{\o}rup
arXiv Machine Learning
Sep 11

Sublinear Variational Optimization of Gaussian Mixture Models with Millions to Billions of Parameters

The paper introduces a highly efficient variational approximation for Gaussian Mixture Models (GMMs) with arbitrary covariances, integrated with mixtures of factor analyzers. This method reduces the per‑iteration runtime from ≠O(NCD^2) to a complexity that scales linearly with dimensionality D and sublinearly with the product NC. Experiments demonstrate sublinear scaling across the entire optimization, order‑of‑magnitude speed‑ups on large benchmarks, training of GMMs with over 10 billion parameters in under nine hours on a single CPU, and competitive zero‑shot image denoising performance.

By Sebastian Salwig, Till Kahlke, Florian Hirschberger, Dennis Forster, J\"org L\"ucke
arXiv Machine Learning
Sep 24

FastManly: An EM-Gradient Algorithm for Manly Mixture Models

FastManly is a new implementation of mixtures of Manly transformations that replaces the Nelder–Mead optimization used in traditional EM algorithms with Newton’s method. The authors derive both the gradient and full Hessian for the EM–gradient algorithm. Simulation studies demonstrate that FastManly achieves improved performance and noticeable speedups compared to the conventional approach.

By Katharine M. Clark, Paul D. McNicholas