arXiv:2608. 12146v1 Announce Type: cross Abstract: Training Mixture-of-Experts (MoE) models for reinforcement learning (RL) couples two load-balancing problems: sequence composition determines dense attention work in each data-parallel microbatch, while token routing determines sparse expert work on expert-parallel ranks.
arXiv:2606. 00395v1 Announce Type: cross Abstract: Mixture of Experts (MoE) Large Language Models (LLMs) achieve strong performance at scale.
arXiv:2610.07332v1 Announce Type: new Abstract: Long-horizon LLM agents are frequently implemented using sparse mixture-of-experts (MoE) models, yet the co-design of agentic behavior and MoE structur...
The paper introduces Ban&Pick, a post‑training, plug‑and‑play routing strategy for Sparse Mixture‑of‑Experts large language models. It identifies and reinforces a small group of highly influential experts while dynamically pruning redundant ones, leading to accuracy gains across math, code, and reasoning benchmarks. Experiments on DeepSeek and Qwen3 show notable performance improvements and a 1.25× inference speedup without retraining or architectural changes.
RoutePack is a replay‑guided planner that coordinates expert placement and rollout‑trajectory scheduling for Mixture‑of‑Experts reinforcement learning models. By incrementally adjusting expert placement while preserving residency and then rescheduling trajectories into token‑capped micro‑batches, it jointly balances dense‑side work and layer‑wise expert peaks under fixed capacity constraints. Evaluations on 7.9B and 124B hybrid‑attention MoE models across NVIDIA H20‑3e configurations show throughput gains of 4.44‑11.85% over a DP‑local first‑fit‑decreasing baseline and 1.33‑5.28% over expert rerouting alone.
MOER is a mix‑policy optimization framework that blends expert guidance with self‑exploration in reinforcement learning. It injects a decaying expert chain‑of‑thought prefix into prompts, progressively annealing the prefix length to encourage exploration, and uses a selective rollback mechanism to replace failed completions with the full expert trace, ensuring correctness. The method employs a harmonized mix‑policy objective that optimizes on‑policy tokens with a GRPO loss while supervising both expert prefix and completion tokens with a trust‑region SFT loss, leading to stable training and superior performance on mathematical and general reasoning tasks.
arXiv:2604.23036v2 Announce Type: replace-cross Abstract: Despite MoE models leading many benchmarks, supervised fine-tuning (SFT) for the MoE architectures remains difficult because its router layer...
arXiv:2609.36222v1 Announce Type: new Abstract: Large language models are increasingly expensive to serve. In large-scale serving systems, autoregressive decoding is often bottlenecked by transferrin...
arXiv:2608. 10392v1 Announce Type: new Abstract: Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts.
ACE introduces a training‑free, calibration‑free framework for adaptive expert skipping in Mixture‑of‑Experts LLMs. It combines a Global Spectral Proxy that estimates global transformation capacity with a Router‑Conditioned Refinement that builds expert‑specific direction prototypes, enabling the model to skip low‑contribution experts while always keeping the top‑1 expert. Offline computation of expert statistics leaves only lightweight table lookups during inference, and experiments on three MoE‑based LLMs show ACE outperforms static and dynamic baselines, especially at high skipping ratios.
HARTS is a training system that accelerates agentic reinforcement learning with hybrid‑attention models by sharing prefixes of irregular rollout trees. It uses a two‑level planner to create compression‑oriented microbatches, refines their membership and execution order, and accumulates gradients before a single backward pass through shared prefixes. The system supports distributed parallelism, activation recomputation, and preserves RL objectives while achieving 4.77‑10.45× higher update throughput on large hybrid‑attention MoE models.
The paper introduces Router Drift Control (RDC), a method for mitigating off‑policy degradation in Mixture‑of‑Experts reinforcement learning. RDC uses the disagreement between rollout and training routers to detect stale routing decisions and selectively rejects high‑drift responses, thereby avoiding the gradient bias introduced by Rollout Routing Replay (R3). Experiments show that RDC improves validation accuracy, especially in asynchronous training settings, outperforming both R3 and Truncated Importance Sampling (TIS).