arXiv AI
Muon Is Theoretically Wrong For Convolutions, But Empirically Effective
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Statistics ML
Sep 10
Muon-C: Operator-Aligned Muon for Convolutional Kernels
arXiv:2609.09676v1 Announce Type: cross Abstract: Muon replaces matrix momentum with an approximately orthogonal polar direction, but its geometry depends on the matrix representation. For convolutio...
Hugging Face Trending Papers
Sep 9
Muon-C: Operator-Aligned Muon for Convolutional Kernels
Muon replaces matrix momentum with an approximately orthogonal polar direction, but its geometry depends on the matrix representation. For convolution, standard unfolding describes a local patch map r...
arXiv AI
Jun 9
Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning
arXiv:2604. 09967v2 Announce Type: replace-cross Abstract: Muon has emerged as a promising optimizer for large-scale foundation model pre-training by exploiting the matrix structure of neural network updates through iterative orthogonalization.
arXiv AI
Jul 7
Turbo-Muon: Almost-Orthogonal Pre-Conditioning for Fast Muon Updates
arXiv:2512. 04632v2 Announce Type: replace Abstract: Orthogonality-based optimizers, such as Muon, have recently shown strong performance across large-scale training and community-driven efficiency challenges.
arXiv Machine Learning
Jun 26
DMuon: Efficient Distributed Muon Training with Near-Adam Overhead
arXiv:2606. 27153v1 Announce Type: cross Abstract: Matrix-orthogonalization-based optimizers, exemplified by Muon, have demonstrated strong convergence behavior across a wide range of modern deep learning workloads.
arXiv AI
Sep 30
Normalize-Then-Precondition: A Hierarchical Approach to Marginal Scale and Interaction Geometry for LLM Training
arXiv:2609.36692v1 Announce Type: cross Abstract: Matrix optimizers have emerged as a promising direction, with Muon standing out as a prominent design. Revisiting Muon through its full-Gram represen...