Hugging Face Trending Papers August 3, 2026

CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization

Read the original on Hugging Face Trending Papers →

Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) performance in visual generative modeling, yet their training remains computationally prohibitive. While the recently proposed Momentum Orthogonalization (Muon) optimizer offers a promising alternative to AdamW, its direct application to DiTs yields suboptimal late-stage convergence.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Jun 2

Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition

arXiv:2411. 09816v5 Announce Type: replace Abstract: Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices.

By Cem \"Uy\"uk, Mike Lasby, Mohamed Yassin, Utku Evci, Yani Ioannou

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Aug 4

UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reduction

arXiv:2608. 01298v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) have emerged as a core architecture in generative modeling due to their scalability and adaptability to multimodal tasks.

By Junno Yun, Ya\c{s}ar Utku Al\c{c}alar, Mehmet Ak\c{c}akaya

llms diffusion multimodal safety

arXiv Machine Learning

Jun 26

DMuon: Efficient Distributed Muon Training with Near-Adam Overhead

arXiv:2606. 27153v1 Announce Type: cross Abstract: Matrix-orthogonalization-based optimizers, exemplified by Muon, have demonstrated strong convergence behavior across a wide range of modern deep learning workloads.

By Vincent Chen, Starrick Liu, Regis Cheng, Dance Yang, Shalfun Li, Ryan Yu, Lucy Liang, Hang Su, Roy Gan, Hao Wang, Qian Wang

llms robotics

Hugging Face Trending Papers

Jun 24

Tensorion: A Tensor-Aware Generalization of the Muon Optimizer

Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models. Recent work has shown that exploiting matrix structure can improve optimization dynamics.

computer-vision

arXiv Machine Learning

Jun 25

Tensorion: A Tensor-Aware Generalization of the Muon Optimizer

arXiv:2606. 25975v1 Announce Type: new Abstract: Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models.

By Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko, Sergei Kudriashov, Maxim Rakhuba

computer-vision

arXiv Machine Learning

Jun 2

RT-Lynx: Putting the GEMM Sparsity In a Right Way for Diffusion Models

arXiv:2605. 26632v2 Announce Type: replace Abstract: Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs.

By Xing Cong, Hanlin Tang, Kan Liu, Lan Tao, Lin Qu, Chenhao Xie

llms diffusion efficiency

Related stories

Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition

UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reduction

DMuon: Efficient Distributed Muon Training with Near-Adam Overhead

Tensorion: A Tensor-Aware Generalization of the Muon Optimizer

Tensorion: A Tensor-Aware Generalization of the Muon Optimizer

RT-Lynx: Putting the GEMM Sparsity In a Right Way for Diffusion Models