InterQ‑RL is a new method for multi‑turn agent reinforcement learning that addresses execution mismatch caused by low‑precision quantized rollouts. It corrects the mismatch by applying bounded execution‑prefix adjustments over the policy history, allocating correction strength across action segments within a fixed displacement budget, and does so without extra model forward passes. Experiments on ALFWorld, WebShop, and ScienceWorld with 4B‑30B backbones show that InterQ‑RL achieves the best quantized performance, matching BF16 within 0.7 points and surpassing it in two settings while maintaining or improving learning speed.
arXiv:2610.07767v1 Announce Type: new Abstract: Reinforcement learning (RL) for post-training large language models (LLMs) incurs substantial computation and memory overhead during rollout generation...
arXiv:2607. 15810v1 Announce Type: new Abstract: Rollout generation is a major bottleneck in Reinforcement Learning (RL) for Mixture-of-Experts (MoE) Large Language Models, motivating low-precision rollout acceleration such as FP8.
arXiv:2608. 11152v1 Announce Type: cross Abstract: Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasingly combine rollout workloads across multiple domains and feedback paradigms.
arXiv:2605. 20256v2 Announce Type: replace Abstract: Reinforcement learning has become a cornerstone for aligning and unlocking the reasoning capabilities of large-scale models.
arXiv:2609.13058v1 Announce Type: new Abstract: Reinforcement learning (RL) has become central to post-training of large language models. Recent advances in RL for Mixture-of-Experts (MoE) models hav...
arXiv:2610.14448v1 Announce Type: cross Abstract: Asynchronous rollout improves the efficiency of reinforcement learning with verifiable rewards (RLVR) by reducing under-utilization during long-tail...
HARTS is a training system that accelerates agentic reinforcement learning with hybrid‑attention models by sharing prefixes of irregular rollout trees. It uses a two‑level planner to create compression‑oriented microbatches, refines their membership and execution order, and accumulates gradients before a single backward pass through shared prefixes. The system supports distributed parallelism, activation recomputation, and preserves RL objectives while achieving 4.77‑10.45× higher update throughput on large hybrid‑attention MoE models.
arXiv:2610.07043v1 Announce Type: cross Abstract: Low-precision execution can substantially accelerate reinforcement learning (RL) for large language models, but discrepancies between learner and sam...
arXiv:2609.33848v2 Announce Type: replace Abstract: Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of m...
arXiv:2607. 17299v1 Announce Type: cross Abstract: Long-horizon rollout generation has become the dominant systems bottleneck in agentic reinforcement learning (RL).
arXiv:2605. 20402v3 Announce Type: replace-cross Abstract: MXFP4 arithmetic can dramatically accelerate reinforcement learning (RL) post-training of large language models (LLMs), yet the quantization error introduces severe accuracy degradation.
The paper introduces SORL, a framework that stabilizes off‑policy reinforcement learning for long‑horizon large language model agents. It identifies two key instability sources—token‑level policy granularity mismatches and high‑variance off‑policy updates—and proposes turn‑level importance sampling and clipping‑triggered normalization to align optimization with multi‑turn interactions. Two instantiations, SO‑PPO and SO‑GRPO, are evaluated on open‑domain, multi‑hop, and medical QA benchmarks, as well as on asynchronous RL for mathematical reasoning, showing improved robustness without the need for early stopping or heuristic tuning.