The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2608.24479v1 Announce Type: new Abstract: Massively parallel simulation changes the data regime in which off-policy reinforcement learning (RL) is trained, challenging stabilizers designed for...
The paper introduces Self‑Sparse Speculative Rollouts (S$^3$R), a method that uses sparse attention to propose tokens and then verifies them with full attention, thereby reducing KV‑cache traffic while preserving the dense‑policy distribution for reinforcement learning rollouts. S$^3$R eliminates the need for a separate drafter by sharing policy parameters between drafting and verification, and a KV‑aware scheduler limits cache thrashing to maintain parallelism. Experiments on long‑context mathematical reasoning with the Qwen3 model show that S$^3$R achieves 1.35–1.39× higher end‑to‑end training throughput with comparable task performance.
arXiv:2610.01133v1 Announce Type: cross Abstract: Scaling reasoning typically spends more compute on reinforcement learning (RL) or on inference. We show that a completed RL training history can yiel...
Massively parallel simulation changes the data regime in which off-policy reinforcement learning (RL) is trained, challenging stabilizers designed for data-limited replay. Through controlled experimen...
arXiv:2608. 07719v1 Announce Type: new Abstract: Offline reinforcement learning repeatedly trains policies from a fixed transition pool, making redundant data costly across seeds and hyperparameters, while naive subsampling can remove rare transitions needed for long-horizon credit assignment.
The paper proposes CANOPY, a minimalist reinforcement learning protocol that addresses two common pitfalls—signal starvation and policy drift—in outcome‑only RL for long‑horizon interactive tasks. By scaling same‑task exploration, keeping updates on‑policy, and anchoring updates with KL divergence, CANOPY enables a Qwen3‑14B agent to achieve top leaderboard results on the AppWorld coding benchmark without auxiliary supervision or elaborate scaffolding. The approach also improves performance on SWE‑bench for a Qwen3.5‑9B model.