arXiv Statistics ML
Regularized Estimation and Feature Selection in Mixtures of Generalized Linear Experts
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Aug 3
GEMSS: A Variational Method for Discovering Multiple Sparse Solutions in Classification and Regression Problems
arXiv:2602. 08913v3 Announce Type: replace Abstract: In underdetermined regression and classification problems, multiple feature subsets often yield equivalent predictive performance.
arXiv AI
Jul 7
FedSPM: Routing-Enabled Federated Learning under Dual Heterogeneity via Semiparametric Mixture
arXiv:2607. 04085v1 Announce Type: cross Abstract: Routing-prediction federated learning has emerged as a new paradigm that reframes inter-client heterogeneity as a resource for system-level intelligence: at inference time, the server routes each external query to the best-matched client for prediction.
arXiv Machine Learning
Jul 7
Efficient Cross-Validation for Sparse Linear Regression
arXiv:2306. 14851v5 Announce Type: replace-cross Abstract: Given a high-dimensional covariate matrix and a response vector, ridge-regularized sparse linear regression selects a subset of features that explains the relationship between covariates and the response in an interpretable manner.
arXiv Machine Learning
Oct 2
Understanding Head Geometry and Dynamics in Federated Regression through a Natural Solution Selection Rule: An Unconstrained Feature Model Analysis
The paper investigates how multiple optimal heads in federated averaging affect the global model in multivariate regression with private backbones and a shared linear head. By introducing a natural selection rule—each client returns the head closest to the broadcast head—and analyzing an unconstrained feature model, the authors show that the shared Gram matrix converges to the Bures‑Wasserstein barycenter of the clients’ optimal Gram matrices, which generally differs from the centralized optimum. A one‑time exchange of target means and covariances can correct this gap, restoring the centralized optimal Gram matrix in a single round under exact local optimization.
arXiv Machine Learning
Jun 9
Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency Without Model Sweeps
arXiv:2510. 12744v2 Announce Type: replace-cross Abstract: We develop a unified statistical framework for softmax-gated Gaussian mixture of experts (SGMoE) that addresses three long-standing obstacles in parameter estimation and model selection: (i) non-identifiability of gating parameters up to common translations, (ii) intrinsic gate-expert interactions that induce coupled differential relations in the likelihood, and (iii) the tight numerator-denominator coupling in the softmax-induced conditional density.
arXiv Machine Learning
Aug 26
A Heterogeneous Mixture of Experts Framework for Interpretable Machine Learning
arXiv:2608.24195v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models provide a flexible framework for partitioning complex prediction problems into simpler local learning tasks through a...