arXiv AI
Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 16
Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act
The paper investigates how reinforcement learning can cause large language model agents to adopt shortcut policies for tool use, relying on superficial prompt cues rather than actual task needs. By creating synthetic environments that mix factual QA and math reasoning, the authors show that agents often invoke tools when cues are present, even when those tools are unnecessary, with spurious invocation rates rising up to 39%. They find that shortcut learning occurs mainly when agents have already mastered the target tool and that semantic alignment between cues and tools amplifies the effect. To counter this, they propose a dense, decision-level reward where an LLM judge assesses tool necessity, which reduces cue-driven tool use while maintaining performance.
arXiv AI
Jun 3
Tool-Aware Optimization with Entropy Guidance for Efficient Agentic Reinforcement Learning
arXiv:2606. 03762v1 Announce Type: cross Abstract: Agentic reinforcement learning (RL) equips large language models (LLMs) with tool-use capabilities that substantially improve reasoning on complex tasks.
arXiv Machine Learning
Jun 8
AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
arXiv:2512. 13278v2 Announce Type: replace-cross Abstract: Agentic reinforcement learning has advanced large language models (LLMs) to reason through long chain-of-thought trajectories while interleaving external tool use.
arXiv AI
Oct 2
Reducing Cognitive Overhead in Tool Use via Multi-Small-Agent Reinforcement Learning
The paper introduces MSARL, a Multi‑Small‑Agent Reinforcement Learning framework that separates reasoning from tool use. A single Reasoning Agent plans and decomposes tasks, while multiple specialized Tool Agents handle specific external tools, each trained with imitation learning and reinforcement learning using role‑specific rewards. Experiments on mathematical problem solving with code execution show that MSARL improves reasoning stability and final‑answer accuracy compared to single‑agent baselines, and the architecture generalizes to other tool‑use tasks.
arXiv AI
Oct 12
Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits
The paper introduces Rollout-Time Intervention (RTI), a method that uses reward-side evidence during rollouts to transform unsuccessful trajectories into successful ones, thereby providing learning signals for tasks beyond a policy’s intrinsic exploration limits. RTI filters out trajectories with excessive distributional shifts, allowing these off-policy interventions to guide policy optimization effectively. Experiments on five benchmarks show that RTI consistently outperforms existing RL baselines, enabling post-trained policies to approach the performance of stronger large language models and surpass their original capability boundaries.
arXiv AI
Jun 10
Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals
arXiv:2606. 10684v1 Announce Type: cross Abstract: Modern language agents which perform multi-step reasoning have shown strong performance in knowledge-intensive question answering.