arXiv Machine Learning
Long-Horizon RL Needs Muon
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
Hugging Face Trending Papers
Jul 8
Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
Reinforcement learning (RL) is becoming increasingly important for post-training large language models (LLMs). Previous RL pipelines for LLMs were mostly synchronous and batch-interleaved, which is inefficient for long-horizon agentic tasks.
arXiv AI
Jul 9
Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
arXiv:2607. 07508v1 Announce Type: cross Abstract: Reinforcement learning (RL) is becoming increasingly important for post-training large language models (LLMs).
arXiv AI
Jul 20
Process Reward Informed Tree Rollout for Effective Multi-Turn RL
arXiv:2607. 15610v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become a key approach for training LLM agents, yet popular methods such as GRPO/RLOO rely on multiple independently sampled complete trajectories for advantage estimation.
arXiv AI
Oct 12
Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits
The paper introduces Rollout-Time Intervention (RTI), a method that uses reward-side evidence during rollouts to transform unsuccessful trajectories into successful ones, thereby providing learning signals for tasks beyond a policy’s intrinsic exploration limits. RTI filters out trajectories with excessive distributional shifts, allowing these off-policy interventions to guide policy optimization effectively. Experiments on five benchmarks show that RTI consistently outperforms existing RL baselines, enabling post-trained policies to approach the performance of stronger large language models and surpass their original capability boundaries.
arXiv AI
Jun 9
Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models
arXiv:2606. 08446v1 Announce Type: cross Abstract: Despite being powerful, reinforcement learning with verifiable rewards (RLVR) induces extremely long COT, making it computationally expensive.
arXiv AI
Jun 10
HIPIF: Hierarchical Planning and Information Folding for Long-Horizon LLM Agent Learning
arXiv:2606. 10507v1 Announce Type: new Abstract: While Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents across a wide range of tasks, their performance often degrades in multi-turn long-horizon agentic tasks.