arXiv Machine Learning July 15, 2026 By Jueun Kim, Baekrok Shin, Jihun Yun, Beomhan Baek, Minhak Song, Chulhee Yun

AMUSE: Anytime Muon with Stable Gradient Evaluation

Read the original on arXiv Machine Learning →

arXiv:2605. 22432v2 Announce Type: replace Abstract: Modern deep learning commonly relies on AdamW with prescribed learning rate schedules, but recent works challenge both components: Schedule-Free optimization removes explicit schedules via iterate averaging, and Muon improves the update geometry by orthogonalizing momentum for matrix parameters.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jul 21

Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization

arXiv:2605. 29547v2 Announce Type: replace-cross Abstract: Deep learning optimization relies heavily on the assumption of smooth loss landscapes, a condition systematically violated by modern architectures due to non-smooth components such as ReLU activations and quantization operators.

By Ruoran Xu, Borong She, Xiaobo Jin, Qiufeng Wang

efficiency

arXiv Machine Learning

Jun 26

DMuon: Efficient Distributed Muon Training with Near-Adam Overhead

arXiv:2606. 27153v1 Announce Type: cross Abstract: Matrix-orthogonalization-based optimizers, exemplified by Muon, have demonstrated strong convergence behavior across a wide range of modern deep learning workloads.

By Vincent Chen, Starrick Liu, Regis Cheng, Dance Yang, Shalfun Li, Ryan Yu, Lucy Liang, Hang Su, Roy Gan, Hao Wang, Qian Wang

llms robotics

arXiv AI

Jun 4

Why Muon Outperforms Adam: A Curvature Perspective

arXiv:2606. 04662v1 Announce Type: cross Abstract: Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear.

By Shuche Wang, Fengzhuo Zhang, Jiaxiang Li, Dirk Bergemann, Zhuoran Yang

safety

arXiv Machine Learning

Jun 9

Lost in the Non-convex Loss Landscape: How to Fine-tune the Large Time Series Model?

arXiv:2606. 08578v1 Announce Type: new Abstract: Recently, large time series models (LTSMs) have gained increasing attention due to their similarities to large language models, including flexible context length, scalability, and task generality, outperforming advanced task-specific models.

By Xu Zhang, Peang Wang, Wei Wang

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 30

To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters

arXiv:2603. 00742v2 Announce Type: replace Abstract: While Adam has long been the ubiquitous default optimizer for deep neural networks, Muon has recently seen rapid adoption due to its superior training speed.

By Sara Dragutinovi\'c, Yedi Zhang, Rajesh Ranganath

safety

arXiv AI

Jun 2

Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling

arXiv:2606. 00888v1 Announce Type: cross Abstract: Dynamic Sparse Training (DST) offers a promising paradigm for improving the training and inference efficiency of deep neural networks; however, we find that in large language model training, DST can suffer from optimization instability, manifested as loss spikes after topology updates.

By Qiao Xiao, Boqian Wu, Patrik Okanovic, Tomasz Sternal, Maurice van Keulen, Elena Mocanu, Mykola Pechenizkiy, Decebal Constantin Mocanu, Torsten Hoefler

llms efficiency

Related stories

Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization

DMuon: Efficient Distributed Muon Training with Near-Adam Overhead

Why Muon Outperforms Adam: A Curvature Perspective

Lost in the Non-convex Loss Landscape: How to Fine-tune the Large Time Series Model?

To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters

Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling