arXiv Statistics ML
Subzero matrix completion for sparse data analysis: large-scale learning of latent low-rank structure
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Aug 11
Understanding Alternating Minimization for Matrix Completion
arXiv:1312. 0925v4 Announce Type: replace Abstract: Alternating Minimization is a widely used and empirically successful heuristic for matrix completion and related low-rank optimization problems.
arXiv Machine Learning
Sep 21
Nonnegative Matrix Factorization in the Component-Wise L1 Norm for Sparse Data
arXiv:2603.29715v2 Announce Type: replace Abstract: Nonnegative matrix factorization (NMF) approximates a nonnegative matrix, X, by the product of two nonnegative factors, WH, where W has r columns a...
arXiv Machine Learning
Aug 27
Maximum-Volume Nonnegative Matrix Factorization
arXiv:2602. 04795v3 Announce Type: replace Abstract: Nonnegative matrix factorization (NMF) is a popular data embedding technique.
arXiv Machine Learning
Jul 1
Direction-Magnitude Decomposition for Low-Rank Matrix Optimization: Faster Convergence and Saddle-to-saddle Dynamics
arXiv:2606. 31390v1 Announce Type: cross Abstract: Low-rank matrix optimization is often carried out via the Burer-Monteiro (BM) formulation, but choosing the factorization rank $r$ is delicate and can substantially slow optimization.
arXiv Machine Learning
Oct 9
D-SLR: The Disjoint Row-Sparse plus Low-Rank Decomposition
The paper introduces D‑SLR, a closed‑form decomposition that combines a low‑rank approximation with a disjoint row‑sparse component, offering a drop‑in replacement for truncated SVD. Unlike previous joint methods, D‑SLR restricts each row to either be stored verbatim or approximated by the low‑rank fit, achieving the joint optimum without extra parameters and without iterative solvers or regularization tuning. The authors provide an assumption‑free, a‑posteriori error bound for every shape, and demonstrate performance gains on synthetic data, LLM embedding tables, network traffic, and hyperspectral images.
arXiv Machine Learning
Sep 18
Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training
The paper introduces low‑rank orthogonalization, a technique that exploits the low‑rank nature of gradients in neural network training to perform matrix orthogonalization more efficiently. Building on this, the authors present low‑rank matrix‑signed gradient descent (MSGD) and a low‑rank variant of the Muon optimizer, showing through experiments that low‑rank Muon matches or surpasses vanilla Muon on GPT‑2 and LLaMA pretraining, especially for larger models. Theoretical analysis provides iteration‑complexity bounds for both low‑rank MSGD and low‑rank Muon under heavy‑tailed noise.