arXiv:2608. 04788v1 Announce Type: cross Abstract: Large language model agents are commonly trained through reinforcement learning with sparse trajectory-level rewards, which offer limited guidance on how strongly individual tokens should be updated.
Large language model agents are commonly trained through reinforcement learning with sparse trajectory-level rewards, which offer limited guidance on how strongly individual tokens should be updated. On-Policy Self-Distillation (OPSD) addresses this by re-scoring generated tokens under a privileged replay view to obtain dense, token-level supervision.
The paper introduces VHD-Play, a pipeline that first samples and solves a mathematical model before generating agentic reinforcement learning environments, ensuring that dynamics and evaluation are aligned from the outset. This approach yields 3,300 diverse environments at a low cost and significantly improves the performance of a large language‑model agent (Qwen3.6‑35B‑A3B) across multiple diagnostic families and external benchmarks. The study demonstrates that stateful interaction is a key factor in learning gains and that scaling the training substrate can further enhance performance.
arXiv:2609.21423v1 Announce Type: new Abstract: Online agent deployments produce abundant execution traces, while task-specific verification and expert annotation are costly to scale. We study how to...
Language-model agents increasingly face long-horizon tasks with evolving state, interdependent decisions, and delayed outcomes. Scaling their training requires diverse agentic environments, dependable...
The paper introduces ReTRAIL, a framework that allows long‑horizon agents to rewind the environment to earlier states while preserving forward progress in knowledge. By training a shared policy with State‑Anchored GRPO and a branch‑advancement reward, ReTRAIL enables agents to recover from unproductive states and reuse insights from abandoned exploration. Experiments on MineSweeper, Sokoban, ALFWorld, and ScienceWorld show that this combined rewinding and experience‑reuse strategy outperforms simple rewind primitives and improves long‑horizon task solving.
arXiv:2607. 16204v1 Announce Type: new Abstract: Recent growth in reinforcement learning (RL) has surfaced a need for diverse, specialized training environments.
arXiv:2510.15047v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) as agents often fail to improve in new environments. We identify and characterize a failure mode we call explora...
HINT-SD introduces a targeted self‑distillation framework for long‑horizon language‑model agents that uses full‑trajectory hindsight to identify failure‑relevant actions and applies feedback‑conditioned distillation only to those action spans. This selective approach reduces the need for per‑turn feedback, improving training efficiency and effectiveness. Experiments on BFCL v3 and AppWorld demonstrate that HINT‑SD outperforms dense per‑turn feedback baselines by up to 13.60 percentage points on average while cutting training time per step by 2.26×.
The paper introduces CoPE, a method that trains language agents by adding multi‑step action targets to the policy backbone. By supervising each action forecasted from preceding histories, CoPE enables acting and forecasting to co‑evolve without extra inference cost. Across five benchmarks, CoPE outperforms existing RL baselines, achieving up to twice the sample efficiency on ScienceWorld and improving overall performance on realistic applications.
arXiv:2608. 06197v1 Announce Type: new Abstract: Training large language model agents for long-horizon tool use typically relies on interactions with real or synthesized executable environments, whose construction and verification are costly, or on external simulators that are difficult to ground.
GRADUS introduces a method for training large language model agents in policy-constrained tool environments by decoupling task design from solutions. It treats a fixed environment as a latent syllabus, extracting guarded behavioral operators and composing them into progressively harder tasks while using corpus-level feedback to target missing outcome combinations. The approach jointly calibrates tasks and solutions against the environment, turning verification into a recovery signal that preserves difficult examples, leading to significant performance gains on benchmarks such as $ au^2$-Bench and CAR-Bench.