arXiv Machine Learning

Decentralized EM Algorithm for Gaussian Mixtures under Data Heterogeneity and Partial Labeling

arXiv:2411. 05591v2 Announce Type: replace-cross Abstract: We systematically study several network-based Expectation-Maximization (EM) algorithms for the Gaussian mixture model within decentralized federated learning (DFL).

arXiv Machine Learning
Aug 19

Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixtures

arXiv:2506. 06584v2 Announce Type: replace Abstract: Learning Gaussian Mixture Models (GMMs) is a fundamental problem in statistics and machine learning, with the Expectation-Maximization (EM) algorithm and its popular variant gradient EM being arguably the most widely used algorithms in practice.

By Mo Zhou, Weihang Xu, Maryam Fazel, Simon S. Du
arXiv Machine Learning
Jun 19

Variational Consensus Monte Carlo for Bayesian Mixture

arXiv:2606. 19643v1 Announce Type: cross Abstract: Motivated by the privacy, sensitivity and sharing limitations of health data, we present a comprehensive pipeline for inference of Bayesian mixture models within a federated learning setting, i.

By Julie Fendler, Francesca L. Crowe, Tom Marshall, Sylvia Richardson, Paul D. W. Kirk
arXiv Machine Learning
Aug 24

Amortized Bandwidth Learning for Kernel Density Estimation under Logarithmic Score

The paper introduces an amortized learning framework for selecting bandwidths in kernel density estimation by optimizing the logarithmic score across a distribution of tasks. It uses a truncated-and-renormalized bounded-support formulation and affine standardization to achieve stable learning and transferability across different intervals. Experiments on Gaussian samples, a multi-family benchmark, and randomized Gaussian mixtures demonstrate that the learned selector outperforms traditional methods such as Silverman’s rule, Sheather–Jones, and least‑squares cross‑validation, especially for small or heterogeneous samples.

By Junyi Liang, Hailiang Du
arXiv Machine Learning
Jun 10

FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection

arXiv:2604. 24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments.

By Yutong He, Zhengyang Huang, Jiahe Geng, Kun Yuan
arXiv Machine Learning
Aug 28

Decentralized Multitask Learning over Learned Task Graphs

The paper presents a decentralized multitask learning framework that learns task relationships directly from distributed data. It introduces a two‑phase strategy: first estimating a generalized graph Laplacian from noisy stochastic gradient iterates, then using the learned graph to facilitate cooperative multitask diffusion learning. The authors provide theoretical analysis of Laplacian estimation error, its impact on steady‑state performance, and a topology sensitivity index, and confirm the benefits of learned task graphs through simulations.

By Zirui Wan, Stefan Vlaski
arXiv Machine Learning
Sep 18

Federated Soft Clustering via Generalized Total Variation Minimization

The paper introduces federated soft clustering for devices in a federated learning network, each fitting a personalized Gaussian mixture model. It proposes Generalized Total Variation Minimization (GTVMin) to couple local maximum likelihood problems via a graph regularizer that penalizes discrepancies between connected nodes’ models. Three discrepancy measures are compared: a squared Euclidean distance requiring component matching, a Monte‑Carlo approximated Kullback‑Leibler divergence, and a closed‑form maximum mean discrepancy; all are optimized with synchronous projected gradient updates, with a convergence guarantee for the smooth MMD instance.

By Shamsiiat Abdurakhmanova, Alexander Jung
arXiv Statistics ML
Sep 14

A data-driven Fourier-mixture neural-network method for density estimation

The paper introduces a data‑driven Fourier‑trained neural‑network estimator for fixed‑horizon probability densities, using a positive Gaussian–Laplace mixture with a closed‑form characteristic function. Training occurs directly in Fourier space, enabling non‑negativity and unit‑mass preservation while handling both i.i.d. and resampling‑based pseudo‑sampling data. The authors provide theoretical error bounds, a multidimensional extension, and demonstrate competitive empirical performance against Expectation–Maximization, especially on heavy‑tailed targets.

By Duy-Minh Dang, Volter Entoma