Hugging Face Trending Papers
Beyond Magnitude: Contrastive Routing for Modular Mixture-of-Experts
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computation and Language
Sep 2
Beyond Magnitude: Contrastive Routing for Modular Mixture-of-Experts
arXiv:2609.01100v1 Announce Type: new Abstract: In current Mixture-of-Experts architectures, routing is performed based on representations dominated by structure shared across all tokens, limiting ex...
arXiv AI
Jun 2
ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
arXiv:2606. 01509v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models scale by activating only a small subset of experts per token.
arXiv Machine Learning
Aug 12
Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation
arXiv:2608. 10392v1 Announce Type: new Abstract: Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts.
arXiv Machine Learning
Jun 17
MoSE: Mixture of Slimmable Experts for Efficient and Adaptive Language Models
arXiv:2602. 06154v2 Announce Type: replace Abstract: Mixture-of-Experts (MoE) models scale large language models efficiently by sparsely activating experts, but once an expert is selected, it is executed fully.
arXiv Computer Vision
3d ago
DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inference
arXiv:2609.38823v1 Announce Type: new Abstract: Multimodal mixture-of-experts (MoE) models combine sparse expert activation with visual-language capabilities, yet their inference remains costly becau...
arXiv Machine Learning
Sep 4
Towards a Statistical Understanding of Mixture-of-Experts
The paper presents a statistical framework for Mixture-of-Experts (MoE) models, treating them as localized aggregation systems. It derives oracle risk bounds that separate approximation, expert‑learning, and router‑estimation errors for both dense and sparse routing with evolving experts. The authors also analyze how sparse Top‑K routing balances computational cost with performance, interpret gating geometrically, and explain how shared experts can capture common predictive structure while allowing routed experts to focus on local residuals.