RoutePack is a replay‑guided planner that coordinates expert placement and rollout‑trajectory scheduling for Mixture‑of‑Experts reinforcement learning models. By incrementally adjusting expert placement while preserving residency and then rescheduling trajectories into token‑capped micro‑batches, it jointly balances dense‑side work and layer‑wise expert peaks under fixed capacity constraints. Evaluations on 7.9B and 124B hybrid‑attention MoE models across NVIDIA H20‑3e configurations show throughput gains of 4.44‑11.85% over a DP‑local first‑fit‑decreasing baseline and 1.33‑5.28% over expert rerouting alone.
arXiv:2608. 12146v1 Announce Type: cross Abstract: Training Mixture-of-Experts (MoE) models for reinforcement learning (RL) couples two load-balancing problems: sequence composition determines dense attention work in each data-parallel microbatch, while token routing determines sparse expert work on expert-parallel ranks.
arXiv:2609.13058v1 Announce Type: new Abstract: Reinforcement learning (RL) has become central to post-training of large language models. Recent advances in RL for Mixture-of-Experts (MoE) models hav...
arXiv:2608. 11152v1 Announce Type: cross Abstract: Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms.
arXiv:2610.07332v1 Announce Type: new Abstract: Long-horizon LLM agents are frequently implemented using sparse mixture-of-experts (MoE) models, yet the co-design of agentic behavior and MoE structur...
Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms. Prefix-aware routing improves inference efficiency through cache reuse and load balancing, but it does not control how heterogeneous rollout sessions compete for KV-cache capacity.
The paper introduces RelayMoE, a ring‑based execution model for Mixture‑of‑Experts (MoE) training that reduces memory usage by avoiding full top‑k expanded dispatch buffers. RelayMoE dynamically chooses between expert and token routing based on communication volume and overlaps data transfer with computation, enabling efficient backward recomputation. Experiments on 30B–57B MoE models show up to a 2× speedup over Megatron‑LM and a 2.02× throughput improvement while extending trainable sequence lengths by up to 2.85×.
ReFlow is a system‑algorithm co‑designed framework for efficient reinforcement learning (RL) post‑training of large language models (LLMs). It introduces replay‑driven decoupling, allowing recent trajectories to be reused multiple times to amortize rollout costs while limiting policy staleness. The architecture features a dual replay cycle, an elastic resource pool, and an adaptive scheduler, achieving up to 2.06× higher training throughput compared to existing synchronous and asynchronous RL systems without sacrificing model performance.
arXiv:2606. 04101v1 Announce Type: cross Abstract: Large-scale expert parallelism (EP) is becoming pivotal for training and serving frontier MoE models, but it also amplifies device-level expert load imbalance into compute stragglers, token all-to-all bottlenecks, and activation-memory spikes.
arXiv:2606. 19004v1 Announce Type: cross Abstract: Reinforcement learning (RL) post-training of Diffusion Transformers (DiTs) is prohibitively expensive, requiring thousands of high-end GPUs.
The paper introduces T1, a 122‑billion‑parameter Mixture‑of‑Experts model trained with reinforcement learning to perform long‑horizon terminal tasks such as coding and scientific discovery. T1 operates a real shell in a cloud sandbox, making over 300 tool‑call turns per task and receiving rewards from task‑specific verifiers. The authors detail a training recipe that includes aggressive warm‑starting, TITO construction with drift repair, and rollout‑routing replay, achieving significant performance gains on Terminal‑Bench 2.1 and surpassing GPT‑5.4 and GLM‑5.1 on the Long‑Horizon Terminal Bench.
arXiv:2609.36830v1 Announce Type: new Abstract: Fully asynchronous reinforcement learning (RL) improves resource utilization in large language model post-training by overlapping rollout generation wi...
arXiv:2607. 05272v1 Announce Type: cross Abstract: Inference serving systems must balance throughput and latency under bursty, heterogeneous workloads, yet the industry standard remains static batching policies that require manual tuning and cannot adapt to shifting traffic.