arXiv AI
Jun 16
PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents
arXiv:2606. 16215v1 Announce Type: cross Abstract: Multi-turn tool-use agents must reason, call tools, and adapt to observations across several interaction turns.
HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning
Related stories
arXiv AI
Jun 30
SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Models
arXiv:2601. 03555v3 Announce Type: replace Abstract: Training reliable tool-augmented agents remains a significant challenge, largely due to the difficulty of credit assignment in multi-step reasoning.
arXiv AI
Jul 23
In-the-Flow Agentic System Optimization for Effective Planning and Tool Use
arXiv:2510. 05592v2 Announce Type: replace Abstract: Outcome-driven reinforcement learning has advanced reasoning in large language models (LLMs), but prevailing tool-augmented approaches train a single, monolithic policy that interleaves thoughts and tool calls under full context; this scales poorly with long horizons and diverse tools and generalizes weakly to new scenarios.
arXiv AI
Jun 2
Learning When Not to Act: Mitigating Tool Abuse in Agentic Reinforcement Learning
arXiv:2606. 02132v1 Announce Type: new Abstract: Agentic reinforcement learning can induce tool abuse, where models overuse external tools even for queries solvable by internal reasoning.
arXiv AI
Jun 3
Tool-Aware Optimization with Entropy Guidance for Efficient Agentic Reinforcement Learning
arXiv:2606. 03762v1 Announce Type: cross Abstract: Agentic reinforcement learning (RL) equips large language models (LLMs) with tool-use capabilities that substantially improve reasoning on complex tasks.
arXiv AI
Oct 2
Reducing Cognitive Overhead in Tool Use via Multi-Small-Agent Reinforcement Learning
The paper introduces MSARL, a Multi‑Small‑Agent Reinforcement Learning framework that separates reasoning from tool use. A single Reasoning Agent plans and decomposes tasks, while multiple specialized Tool Agents handle specific external tools, each trained with imitation learning and reinforcement learning using role‑specific rewards. Experiments on mathematical problem solving with code execution show that MSARL improves reasoning stability and final‑answer accuracy compared to single‑agent baselines, and the architecture generalizes to other tool‑use tasks.
arXiv AI
Sep 2
ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning
ARISE‑RL is a full‑cycle self‑evolution framework that couples a task/rubric generator with a reasoning solver, enabling open‑ended agents to learn via reinforcement learning without relying on gold answers. The generator creates tool‑grounded rubric criteria and rewards itself for producing valid, intermediate‑difficulty tasks, while the solver improves through fine‑grained rubric satisfaction signals and multi‑step reasoning. The framework also introduces Reward‑Gated Self‑Evolution Distillation to selectively distill memory‑augmented policies, reducing distribution mismatch, and it is evaluated on the new ECR‑Bench rubric suite, achieving state‑of‑the‑art performance across all benchmarks.
arXiv AI
Aug 19
PlanPO: Group Planning-Aware Policy Optimization for Multi-Turn Agentic LLMs
PlanPO introduces a group planning-aware policy optimization method for multi-turn agentic large language models, addressing the issue of advantage collapse caused by treating all successful trajectories equally. By incorporating coarse-to-fine advantage signals that reflect differences in trajectory and turn lengths, PlanPO encourages agents to learn generalizable planning and generation behaviors. Experiments show a 27.2% average improvement over GRPO on benchmarks such as ALFWorld, WebShop, and SciWorld, with minimal extra training cost.
arXiv AI
Aug 5
TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning
arXiv:2608. 04007v1 Announce Type: cross Abstract: Tool-Integrated Reasoning (TIR) enables LLMs to solve complex tasks through iterative tool interactions.
arXiv AI
Jun 2
RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
arXiv AI
Oct 12
Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits
The paper introduces Rollout-Time Intervention (RTI), a method that uses reward-side evidence during rollouts to transform unsuccessful trajectories into successful ones, thereby providing learning signals for tasks beyond a policy’s intrinsic exploration limits. RTI filters out trajectories with excessive distributional shifts, allowing these off-policy interventions to guide policy optimization effectively. Experiments on five benchmarks show that RTI consistently outperforms existing RL baselines, enabling post-trained policies to approach the performance of stronger large language models and surpass their original capability boundaries.
arXiv AI
Jul 20
ToolVerse: Unlocking Massive Environments and Long-Horizon Tasks for Agentic Reinforcement Learning
arXiv:2607. 15660v1 Announce Type: new Abstract: While LLM agents demonstrate strong reasoning abilities in compact and well-defined scenarios, they struggle to maintain robustness and effectiveness when faced with large-scale, diverse, and dynamic real-world environments that demand seamless tool integration.