The paper introduces TASPO, a method that transforms privileged information (PI) into outcome‑grounded action credit for language‑model agents. TASPO constructs decision‑applicable PI from verified successful experience, aggregates PI‑induced likelihood shifts at the executable‑action level, and converts relative action support into positive, bounded, mean‑preserving weights on the original trajectory advantage. Experiments on three agentic benchmarks show TASPO improves over GRPO by 10.6% and generalizes better to unseen tasks, while reducing supervision mismatch and stabilizing policy optimization.
By Jingxiao Yang, Wangjie Gan, Yingxuan Zhuang, Wenqi Zhang, Jintao Chen, Xuhong Zhang
arXiv:2608. 16156v1 Announce Type: new Abstract: Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult.
By Huan Zhang, Mingju Chen, Dongxu Zhou, Can Lv, Heng Chang, Sen Cui, Faguo Wu, Shiji Zhou
Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult. Existing approache...
The paper introduces CrEST, a hierarchical credit assignment framework for multi-turn, multi-step large language model agents. CrEST anchors each token’s advantage sign to verifier-defined rewards while using a self-teacher to provide dense magnitude signals, addressing both inter-turn dilution and intra-turn token contribution issues. Experiments on BFCL V3 and WildToolBench show CrEST consistently outperforms reinforcement learning and distillation baselines, especially on long trajectories and strict session-level metrics.
By Zechuan Wang, Siyuan Lu, Hongxuan Zhang, Linjian Mo, Chenyi Zhuang, Leilei Gan
arXiv:2606. 32017v1 Announce Type: cross Abstract: Agentic reinforcement learning requires assigning credit to environment-facing actions such as searches, clicks, edits, navigation commands, and object interactions.
By Yuanda Xu, Zhengze Zhou, Hejian Sang, Xiaomin Li, Jiaxin Zhang, Xinchen Du, Zhipeng Wang, Alborz Geramifard
arXiv:2608. 05987v1 Announce Type: new Abstract: Reinforcement learning (RL) with verifiable rewards constructs trajectory-level advantage estimates, yet it often fails to credit the few pivotal decisions that determine outcomes in long-horizon, multi-turn agentic tasks.
By Zi-Han Wang, Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Jie Wu, Zhengzhou Cai, Yueqing Sun, Ziang Ye, Linji Hao, Qi Gu, Xunliang Cai, Yongliang Shen, Yujiu Yang