The paper introduces Grounded Credit Assignment (GCA) for reinforcement learning in multi‑turn search agents. GCA separates credit for evidence retrieval and answer correctness, correcting the bias in GRPO that favors evidence‑retrieving rollouts only proportionally to accuracy gains. Applying GCA to Search‑R1 boosts average EM by 5.7–6.3 points and increases the proportion of evidence‑supported correct answers from 76.9% to 93.4%.
By Qirui Liu, Hao Chen, Yi Wen, Weijie Shi, Yao Zhao, Xianhe Chen, Hao Zhao, Xing Tang, Weihong Luo, Yuyu Luo
BiCAA introduces bidirectional credit assignment for search-augmented agents by combining Forward Solvability Gain (FSG) and Subsequent Reasoning Gain (SRG). FSG measures how each search step increases the likelihood of the correct answer, while SRG evaluates how a step supports future reasoning, discouraging redundant retrieval. Experiments on seven open‑domain QA benchmarks and two Qwen backbones show that BiCAA improves average F1, especially on multi‑hop tasks.
By Yibin Huang, Bin Xu, Hailong Cao, Conghui Zhu
arXiv:2610.07349v1 Announce Type: cross
Abstract: Group Relative Policy Optimization (GRPO) avoids a separate critic by estimating advantages from rollout groups. For multi-turn agents, however, traj...
By Sayak Chakrabarti, Sathish Reddy Indurthi
Self-Credit is an agentic reinforcement learning method that transforms sparse outcome rewards into dense supervision signals through self-generated credit assignment and intra-group rescaling. The approach involves the policy model summarizing key evidence with citations, refining these citations by comparing actions across successful and failed rollouts, and reweighting advantages based on the rescaled credit. Experiments on BrowseComp-Plus, GAIA, and xBench-DeepSearch show significant performance gains over baseline models and other agentic RL methods.
By Fangcong Yin, Ming Yin, Tanvi Bhandarkar, Aashu Singh, Eryk Helenowski, Shlok Kumar Mishra, Saghar Hosseini, Karthik Abinav Sankararaman, Greg Durrett, Dinkar Juyal
arXiv:2608. 06128v1 Announce Type: new Abstract: Search agents extend large language models beyond static parametric memory by enabling them to acquire and use ex ternal evidence during multi-step reasoning.
By Xingyu Guo, Wei Chen, Linlin Yang, Baochang Zhang
arXiv:2605. 12969v3 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) is one of the most widely adopted RLVR algorithms for post-training large language models on reasoning tasks.
By Feng Zhang, Xinhong Ma, Ziqiang Dong, Xi Leng, Jianfei Zhao, Xin Sun, Yang Yang, Guanjun Jiang