SmolLM - blazingly fast and remarkably powerful

SmolVLM Grows Smaller – Introducing the 256M & 500M Models!

Make LLM Fine-tuning 2x faster with Unsloth and 🤗 TRL

Rethinking LLM Evaluation with 3C3H: AraGen Benchmark and Leaderboard

When to use what Schatten-$p$ norm in deep learning?

arXiv:2606. 15268v1 Announce Type: new Abstract: Schatten-$\infty$ based optimizers such as Muon have shown promising empirical performance, but there remains seemingly conflicting observations regarding whether they are beneficial.

Incredibly Fast BLOOM Inference with DeepSpeed and Accelerate

LionMuon: Alternating Spectral and Sign Descent for Efficient Training

LionMuon is a new optimizer that alternates between Lion’s sign-based updates and Muon’s spectral matrix-sign updates on a fixed period P, sharing a single dual-EMA momentum buffer. This design keeps the memory footprint the same as Lion and half that of AdamW while reducing the average iteration cost compared to Muon. Experiments on 124M, 355M, and 720M models show LionMuon Pareto-dominates Muon, Lion, Signum, and AdamW across datasets and architectures, achieving lower validation loss with less compute.