AgentRM proposes a generalizable reward model to guide LLM-based agents during test-time search, outperforming direct policy fine-tuning. Three reward modeling strategies—explicit, implicit, and LLM-as-a-judge—are explored, and AgentRM improves base policy performance by an average of 8.8 points across nine tasks, surpassing top general agents by 4.0 points. It also shows strong weak-to-strong generalization and can boost specialized agents, with plans to release code for further research.
By Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, Maosong Sun
arXiv:2608. 05102v1 Announce Type: new Abstract: Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer.
By Yijun Lu, Rui Ye, Jiajun Wang, Yuwen Du, Tian Jin, Songhua Liu, Siheng Chen
arXiv:2610.17666v1 Announce Type: cross
Abstract: Reinforcement learning with verifiable rewards (RLVR) for multi-turn agents relies on sparse terminal rewards that say little about individual steps....
By Xuyan Ye, Xiyan Fu, Shengda Fan, Jiawei Cui, Yankai Lin, Wei Lu
arXiv:2606. 00017v1 Announce Type: new Abstract: Training language model agents for multi-agent strategic interaction presents a core difficulty: the quality of any action may depend on future events that never materialize, on moves that violate game rules, or on decisions made by other players.
By Aliaksei Korshuk, Alexander Buyantuev, Ilya Makarov
arXiv:2608.01822v2 Announce Type: replace
Abstract: Training LLM-based search agents requires high-quality search data: tasks that demand genuine multi-hop retrieval and trajectories that use search...
By Wentao Tan, Qiong Cao, Jiaqi Wang, Nan Duan
arXiv:2609.36641v1 Announce Type: cross
Abstract: Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning....
By Shengda Fan, Xin Cong, Zhong Zhang, Haotian Chen, Yankai Lin