HARTS: Efficient Agentic Reinforcement Learning for Hybrid-Attention Models over Arbitrary Rollout Trees
Read the original on arXiv Machine Learning →HARTS (Hybrid‑Attention RL over Tree Structures) is a new system that jointly plans microbatches, data‑parallel replica assignments, and microbatch‑slot schedules to efficiently train agentic reinforcement learning models with hybrid attention over arbitrary rollout trees. It uses prefix compression and a linear‑time algorithm for chunkwise linear attention to avoid recomputing shared prefixes, enabling activation recomputation and bounded state replay while preserving trajectory‑wise training benefits. In experiments on an Agentic RL workload derived from SWE‑bench tasks, HARTS delivers 4.81–4.87× speedups in forward, backward, and gradient computations across multiple parallel configurations, with numerical differences comparable to baseline self‑rerun variation and a similar reward trend over the first 120 training steps.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.