arXiv Machine Learning

Identifiability and Estimation for Unlabeled Finite Mixtures under Marginal Independence

arXiv:2606. 07914v1 Announce Type: cross Abstract: We study component recovery and mixing-matrix estimation from unlabeled finite mixtures whose observable distributions share the same latent components but have unknown mixing weights.

arXiv Machine Learning
Jun 9

Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency Without Model Sweeps

arXiv:2510. 12744v2 Announce Type: replace-cross Abstract: We develop a unified statistical framework for softmax-gated Gaussian mixture of experts (SGMoE) that addresses three long-standing obstacles in parameter estimation and model selection: (i) non-identifiability of gating parameters up to common translations, (ii) intrinsic gate-expert interactions that induce coupled differential relations in the likelihood, and (iii) the tight numerator-denominator coupling in the softmax-induced conditional density.

By Do Tien Hai, Trung Nguyen Mai, TrungTin Nguyen, Nhat Ho, Binh T. Nguyen, Christopher Drovandi
arXiv Statistics ML
Sep 25

Riemannian Gradient Descent for Gaussian Mixture Models with unknown diagonal covariances

The paper studies the numerical solution of the Beurling‑LASSO (BLASSO) for estimating Gaussian mixture models (GMMs) with unknown numbers of components and unknown diagonal covariance matrices. It introduces a Conic Particle Gradient Descent (CPGD) algorithm that incorporates Riemannian gradient descent to respect the Fisher‑Rao geometry of Gaussian distributions. The authors provide theoretical convergence guarantees, including exponential local convergence under a non‑degeneracy condition related to component separation, and demonstrate through numerical experiments that CPGD is more robust to overspecification of components than the EM algorithm.

By Romane Giard, Yohann De Castro, Roland Denis, Cl\'ement Marteau
arXiv Statistics ML
3d ago

Adaptive mixture variational inference for spike-and-slab regression

The paper introduces an adaptive fitting procedure for mixtures of product distributions in Gaussian regression with a spike‑and‑slab prior, directly minimizing reverse Kullback‑Leibler divergence on inclusion indicators and active coefficients. This method jointly refines component parameters and weights as the mixture grows, avoiding extra divergence penalties on unused latent coefficients. Empirical results on 250 simulated datasets show that mixtures reduce errors in inclusion probabilities, grouped support probabilities, and coefficient covariance compared to multistart mean‑field approaches, and that direct joint refinement outperforms augmented or restricted refinement at fixed mixture size.

By Hanqing Li, Yaroslav Golub, Xuewen Lu
Hugging Face Trending Papers
Jul 15

Linear Independent Component Analysis via Optimal Transport

Linear Independent Component Analysis (ICA) recovers jointly independent source signals from their linear mixtures. To achieve this, classical ICA algorithms attempt to maximize non-Gaussianity, measured by negentropy, which is linked to independence by information theory.