arXiv Computation and Language
Expert-Space Exploration in MoE Reinforcement Learning
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Aug 14
RoutePack: Expert Placement and Attention-Aware Data Packing for MoE Reinforcement Learning
arXiv:2608. 12146v1 Announce Type: cross Abstract: Training Mixture-of-Experts (MoE) models for reinforcement learning (RL) couples two load-balancing problems: sequence composition determines dense attention work in each data-parallel microbatch, while token routing determines sparse expert work on expert-parallel ranks.
arXiv AI
Jun 2
PR2: Predictive Routing Replay for MoE-Based LLM Reinforcement Learning
arXiv:2606. 00395v1 Announce Type: cross Abstract: Mixture of Experts (MoE) Large Language Models (LLMs) achieve strong performance at scale.
arXiv Machine Learning
Oct 7
Structuring MoE Expert Selection for Agentic Reinforcement Learning
arXiv:2610.07332v1 Announce Type: new Abstract: Long-horizon LLM agents are frequently implemented using sparse mixture-of-experts (MoE) models, yet the co-design of agentic behavior and MoE structur...
arXiv Machine Learning
Aug 27
Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing
The paper introduces Ban&Pick, a post‑training, plug‑and‑play routing strategy for Sparse Mixture‑of‑Experts large language models. It identifies and reinforces a small group of highly influential experts while dynamically pruning redundant ones, leading to accuracy gains across math, code, and reasoning benchmarks. Experiments on DeepSeek and Qwen3 show notable performance improvements and a 1.25× inference speedup without retraining or architectural changes.
arXiv Machine Learning
Oct 19
RoutePack: Replay-Guided Expert Placement and Rollout Trajectory Scheduling for MoE Reinforcement Learning
RoutePack is a replay‑guided planner that coordinates expert placement and rollout‑trajectory scheduling for Mixture‑of‑Experts reinforcement learning models. By incrementally adjusting expert placement while preserving residency and then rescheduling trajectories into token‑capped micro‑batches, it jointly balances dense‑side work and layer‑wise expert peaks under fixed capacity constraints. Evaluations on 7.9B and 124B hybrid‑attention MoE models across NVIDIA H20‑3e configurations show throughput gains of 4.44‑11.85% over a DP‑local first‑fit‑decreasing baseline and 1.33‑5.28% over expert rerouting alone.
arXiv Machine Learning
Oct 23
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback
MOER is a mix‑policy optimization framework that blends expert guidance with self‑exploration in reinforcement learning. It injects a decaying expert chain‑of‑thought prefix into prompts, progressively annealing the prefix length to encourage exploration, and uses a selective rollback mechanism to replace failed completions with the full expert trace, ensuring correctness. The method employs a harmonized mix‑policy objective that optimizes on‑policy tokens with a GRPO loss while supervising both expert prefix and completion tokens with a trust‑region SFT loss, leading to stable training and superior performance on mathematical and general reasoning tasks.