arXiv AI
Beyond the Matrix Sign: Quadratic Spectral Descent
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 15
Muon$^p$: Muon with Fractional Spectral Powers
arXiv:2606. 13867v1 Announce Type: new Abstract: Muon is an increasingly widely used optimizer that replaces a gradient $G=USV^\top$ with its polar factor $UV^\top$, thereby flattening the singular spectrum.
arXiv AI
Jul 23
An Isotropy-Preserving Spectral Cap for Muon: Theory and Three Case Studies
arXiv:2607. 19771v1 Announce Type: cross Abstract: Muon and related matrix-sign optimizers are increasingly used to pre-train large language models, but their effect on the internal geometry of individual weight matrices is not well understood.
arXiv AI
Jun 2
DynMuon: A Dynamic Spectral Shaping View of Muon
arXiv:2605. 17109v3 Announce Type: replace-cross Abstract: In recent years, Muon has emerged as the dominant method for training large language models, and transformers more broadly.
arXiv Machine Learning
Jun 9
LiMuon: Light and Fast Muon Optimizer for Large Models
arXiv:2509. 14562v4 Announce Type: replace Abstract: Large models recently are widely applied in machine learning, so efficient training of large models has received widespread attention.
arXiv Machine Learning
Jun 25
Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
arXiv:2606. 25975v1 Announce Type: new Abstract: Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models.
arXiv Machine Learning
Jun 30
Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics
arXiv:2606. 30509v1 Announce Type: new Abstract: Matrix factorization (i.