arXiv AI
Jun 2
DynMuon: A Dynamic Spectral Shaping View of Muon
arXiv:2605. 17109v3 Announce Type: replace-cross Abstract: In recent years, Muon has emerged as the dominant method for training large language models, and transformers more broadly.
Second-Order Muon Done Right: A Principled Marriage of Spectral Geometry and Curvature
arXiv:2608. 09763v1 Announce Type: new Abstract: Muon's polar update is exact for an unweighted spectral geometry.
Related stories
arXiv Machine Learning
Jun 9
The Spectral Dynamics and Noise Geometry of Muon
arXiv:2606. 08388v1 Announce Type: new Abstract: Muon replaces a matrix gradient $G=U\Sigma V^\top$ by its polar factor $UV^\top$.
arXiv AI
Jun 4
Why Muon Outperforms Adam: A Curvature Perspective
arXiv:2606. 04662v1 Announce Type: cross Abstract: Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear.
arXiv AI
Jul 23
An Isotropy-Preserving Spectral Cap for Muon: Theory and Three Case Studies
arXiv:2607. 19771v1 Announce Type: cross Abstract: Muon and related matrix-sign optimizers are increasingly used to pre-train large language models, but their effect on the internal geometry of individual weight matrices is not well understood.
arXiv AI
Jul 7
Turbo-Muon: Almost-Orthogonal Pre-Conditioning for Fast Muon Updates
arXiv:2512. 04632v2 Announce Type: replace Abstract: Orthogonality-based optimizers, such as Muon, have recently shown strong performance across large-scale training and community-driven efficiency challenges.
arXiv Machine Learning
Jun 4
Muon in Associative Memory Learning: Training Dynamics and Scaling Laws
arXiv:2602. 05725v3 Announce Type: replace Abstract: Muon updates matrix parameters via the matrix sign of the gradient and has shown strong empirical gains, yet its dynamics and scaling behavior remain unclear in theory.
arXiv Machine Learning
Jun 15
Muon$^p$: Muon with Fractional Spectral Powers
arXiv:2606. 13867v1 Announce Type: new Abstract: Muon is an increasingly widely used optimizer that replaces a gradient $G=USV^\top$ with its polar factor $UV^\top$, thereby flattening the singular spectrum.
arXiv AI
Jul 16
Reassessing Muon for Matrix Factorization
arXiv:2607. 13246v1 Announce Type: cross Abstract: Muon has recently emerged as a strong optimizer for large-scale deep learning, where it reshapes gradient updates through approximate orthogonalization and has been reported to outperform Adam and AdamW in large language model training.
arXiv Machine Learning
Jul 17
Muse: Representation Geometry of Muon Beyond Normalized Momentum
arXiv:2607. 14536v1 Announce Type: new Abstract: Muon-style optimizers apply a polar map to matrix momentum, but their updates also depend on the representation of each parameter block before orthogonalization.
arXiv Machine Learning
Aug 5
Muon Meets Mamba: Spectral Optimization for State Space Models
arXiv:2608. 03941v1 Announce Type: new Abstract: Muon is a recent optimizer that orthogonalizes the update to each weight matrix with a Newton-Schulz iteration, which performs steepest descent under the spectral norm.
Hugging Face Trending Papers
Jul 14
Reassessing Muon for Matrix Factorization
Muon has recently emerged as a strong optimizer for large-scale deep learning, where it reshapes gradient updates through approximate orthogonalization and has been reported to outperform Adam and AdamW in large language model training. Its empirical success has motivated a growing body of theoretical work that interprets Muon as steepest descent under the spectral norm.
arXiv Machine Learning
Aug 7
Muon on the Stiefel Manifold Admits an Exact Closed-Form Update
arXiv:2608. 06218v1 Announce Type: cross Abstract: We study Muon, a recently proposed matrix-aware optimization method, in the context of the Stiefel manifold.