arXiv AI
Aug 3
PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization
arXiv:2605. 17877v2 Announce Type: replace Abstract: A significant hurdle for current LLMs is the execution of complex, multi-stage tasks.
TAPO: Tool-Aware Policy Optimization via Credit Transfer for Multimodal Search Agents
arXiv:2606. 05784v1 Announce Type: new Abstract: We identify and formally characterize credit misassignment as a systematic failure mode of GRPO in tool-augmented multimodal search agents: its uniform broadcast of trajectory-level advantages to all tokens causes valuable tool-use steps in failing trajectories to be penalized no differently from valueless ones.
Related stories
arXiv Machine Learning
Jul 7
CLEANER: Self-Purified Trajectories Boost Agentic Reinforcement Learning
arXiv:2601. 15141v2 Announce Type: replace Abstract: Agentic Reinforcement Learning (RL) has empowered Large Language Models (LLMs) to utilize tools like Python interpreters for complex problem-solving.
arXiv AI
Jul 7
Agent Reinforcement Learning via Pivotal-Aware Self-Feedback Retry
arXiv:2607. 03702v1 Announce Type: new Abstract: Large language model (LLM) agents have shown strong decision-making capabilities in long-horizon interactive tasks, yet they still struggle to effectively leverage failed trajectories: full retries incur high interaction costs, while experience retrieval tends to dilute critical experience signals.
arXiv AI
Jul 14
STAMP: Provenance-Guided Credit Assignment for Deep Search Agents
arXiv:2607. 11172v1 Announce Type: new Abstract: Reinforcement learning for deep-search agents has largely focused on trajectory-level scoring -- outcome correctness, citation-aware rewards, and evidence coverage.
arXiv AI
Jun 18
RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents
arXiv:2606. 19047v1 Announce Type: new Abstract: Multi-turn tool-use RL is bottlenecked by the rapid depletion of informative samples in static datasets.
arXiv Machine Learning
Jul 16
TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents
arXiv:2607. 13988v1 Announce Type: new Abstract: Multi-turn agents solve complex tasks through extended sequences of tool interactions before producing a final answer, making credit assignment a fundamental challenge during post-training.
arXiv AI
Jun 8
SlimSearcher: Training Efficiency-Aware Web Agents via Adaptive Reward Gating
arXiv:2606. 07074v1 Announce Type: cross Abstract: Deep research agents have demonstrated remarkable capabilities in complex information-seeking tasks, yet this power comes at a steep computational cost.
arXiv AI
Aug 6
ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignment
arXiv:2608. 05102v1 Announce Type: new Abstract: Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer.
arXiv AI
Jun 11
APPO: Agentic Procedural Policy Optimization
arXiv:2606. 12384v1 Announce Type: cross Abstract: Recent advances in agentic Reinforcement Learning (RL) have substantially improved the multi-turn tool-use capabilities of large language model agents.
arXiv AI
Jul 29
CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization
arXiv:2607. 25659v1 Announce Type: new Abstract: Rubric-based reinforcement learning enriches language model training by evaluating model outputs against explicit criteria.
arXiv AI
Jul 10
When Implausible Tokens Get Reinforced: Tail-Aware Credit Calibration for LLM Reinforcement Learning
arXiv:2607. 07976v1 Announce Type: cross Abstract: Reinforcement learning (RL) has achieved remarkable success in enhancing the reasoning capabilities of large language models (LLMs).
arXiv Machine Learning
Jul 9
UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma
arXiv:2607. 06987v1 Announce Type: new Abstract: Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs).