arXiv:1312. 0925v4 Announce Type: replace Abstract: Alternating Minimization is a widely used and empirically successful heuristic for matrix completion and related low-rank optimization problems.
arXiv:2603.29715v2 Announce Type: replace Abstract: Nonnegative matrix factorization (NMF) approximates a nonnegative matrix, X, by the product of two nonnegative factors, WH, where W has r columns a...
arXiv:2602. 04795v3 Announce Type: replace Abstract: Nonnegative matrix factorization (NMF) is a popular data embedding technique.
arXiv:2606. 31390v1 Announce Type: cross Abstract: Low-rank matrix optimization is often carried out via the Burer-Monteiro (BM) formulation, but choosing the factorization rank $r$ is delicate and can substantially slow optimization.
The paper introduces D‑SLR, a closed‑form decomposition that combines a low‑rank approximation with a disjoint row‑sparse component, offering a drop‑in replacement for truncated SVD. Unlike previous joint methods, D‑SLR restricts each row to either be stored verbatim or approximated by the low‑rank fit, achieving the joint optimum without extra parameters and without iterative solvers or regularization tuning. The authors provide an assumption‑free, a‑posteriori error bound for every shape, and demonstrate performance gains on synthetic data, LLM embedding tables, network traffic, and hyperspectral images.
The paper introduces low‑rank orthogonalization, a technique that exploits the low‑rank nature of gradients in neural network training to perform matrix orthogonalization more efficiently. Building on this, the authors present low‑rank matrix‑signed gradient descent (MSGD) and a low‑rank variant of the Muon optimizer, showing through experiments that low‑rank Muon matches or surpasses vanilla Muon on GPT‑2 and LLaMA pretraining, especially for larger models. Theoretical analysis provides iteration‑complexity bounds for both low‑rank MSGD and low‑rank Muon under heavy‑tailed noise.
arXiv:2607. 13919v1 Announce Type: new Abstract: Nonnegative Matrix Factorization (NMF) is a fundamental tool in unsupervised learning, which approximates a nonnegative matrix by the product of two low-rank nonnegative factors.
The paper compares two popular data‑integration techniques—Stack‑SVD, which concatenates datasets before performing singular value decomposition, and SVD‑Stack, which first decomposes each dataset separately and then aggregates the leading singular vectors. By deriving exact asymptotic performance expressions and phase transitions in a proportional regime, the authors show that neither method uniformly dominates the other when unweighted, but optimally weighted Stack‑SVD outperforms optimally weighted SVD‑Stack when the low‑rank signal is fully shared. They also demonstrate that SVD‑Stack can excel with partially shared components and provide practical algorithms for estimating optimal weights, supported by simulations and genomic experiments.
arXiv:2607. 20084v1 Announce Type: cross Abstract: Non--negative matrix factorization (NMF) has become an established dimensionality reduction technique for extracting latent structures from non--negative data and has found widespread applications in fields such as bioinformatics, text mining, image analysis, and recommender systems.
arXiv:2607. 24518v1 Announce Type: new Abstract: Symmetric non-negative matrix factorization (SymNMF) recovers latent group structure from a dependence matrix, but its dense, quadratic-memory objective has confined prior work to moderate sizes.
arXiv:2406. 10407v3 Announce Type: replace-cross Abstract: Semidefinite programs (SDPs) and their solvers are powerful tools with many applications in machine learning and data science.
arXiv:2606. 14040v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are typically trained to reconstruct the \textbf{entire} residual stream through a sparse dictionary, implicitly assuming that all activation content is amenable to sparse, monosemantic decomposition.