Self-Credit is an agentic reinforcement learning method that transforms sparse outcome rewards into dense supervision signals through self-generated credit assignment and intra-group rescaling. The approach involves the policy model summarizing key evidence with citations, refining these citations by comparing actions across successful and failed rollouts, and reweighting advantages based on the rescaled credit. Experiments on BrowseComp-Plus, GAIA, and xBench-DeepSearch show significant performance gains over baseline models and other agentic RL methods.
LLM agents are commonly trained with RLVR, but the sparse nature of typical RLVR rewards makes credit assignment challenging, especially for long-horizon tasks. We propose Self-Credit, an end-to-end a...
The paper introduces Grounded Credit Assignment (GCA) for reinforcement learning in multi‑turn search agents. GCA separates credit for evidence retrieval and answer correctness, correcting the bias in GRPO that favors evidence‑retrieving rollouts only proportionally to accuracy gains. Applying GCA to Search‑R1 boosts average EM by 5.7–6.3 points and increases the proportion of evidence‑supported correct answers from 76.9% to 93.4%.
Dr.Credit introduces a rubric‑grounded credit assignment method that evaluates intermediate tool turns in deep research agents by comparing the information returned to the history of accepted support for each rubric. Unlike traditional approaches that rely on ground‑truth answers, Dr.Credit uses task requirements as a shared reference, distinguishing new support from previously seen evidence and recognizing partial rubric fulfillment. Experiments on four benchmarks show that Dr.Credit outperforms open deep research baselines across all primary metrics, achieving performance competitive with proprietary models while enabling more efficient evidence acquisition and higher‑quality reports under limited turn budgets.
arXiv:2608. 05987v1 Announce Type: new Abstract: Reinforcement learning (RL) with verifiable rewards constructs trajectory-level advantage estimates, yet it often fails to credit the few pivotal decisions that determine outcomes in long-horizon, multi-turn agentic tasks.
arXiv:2608. 05102v1 Announce Type: new Abstract: Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer.
arXiv:2607. 03702v1 Announce Type: new Abstract: Large language model (LLM) agents have shown strong decision-making capabilities in long-horizon interactive tasks, yet they still struggle to effectively leverage failed trajectories: full retries incur high interaction costs, while experience retrieval tends to dilute critical experience signals.
arXiv:2609.36178v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) has become a promising approach for training large language model agents. However, its uniform assignment o...
The paper introduces VICT, a method that leverages the internal structure of verifiable tasks to perform fine‑grained credit assignment for long‑horizon LLM agents. VICT exposes executable or evidence‑backed atoms from a task’s terminal verifier and traces them back to actions via dependency‑valid proof edges, redistributing advantage only along these edges. This approach improves performance on ALFWorld and WebShop compared to outcome‑only training and matches recent fine‑grained credit methods without requiring additional critics, labels, or inference‑time verifier access.
arXiv:2610.19103v1 Announce Type: new Abstract: Group-based reinforcement learning enables large language model agents to learn from sparse outcome rewards in long-horizon tasks. Recent methods deriv...
arXiv:2610.07349v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) avoids a separate critic by estimating advantages from rollout groups. For multi-turn agents, however, traj...
arXiv:2607. 11172v1 Announce Type: new Abstract: Reinforcement learning for deep-search agents has largely focused on trajectory-level scoring -- outcome correctness, citation-aware rewards, and evidence coverage.
arXiv:2610.15840v1 Announce Type: cross Abstract: Reinforcement learning (RL) for large language model (LLM) agents remains challenging in long-horizon tasks with sparse outcome rewards, where assign...