SCOPE is a new framework for selective critical‑step on‑policy self‑distillation in agentic reinforcement learning. After a supervised warm‑up, the policy examines its own trajectories, generates hindsight skills for each step, and distills guidance only at selected error steps, using a bounded teacher‑student coefficient and a success‑relative pairwise length reward. Experiments on ALFWorld and WebShop show consistent performance gains across multiple model backbones, with Qwen2.5‑3B achieving 97.7 % success on ALFWorld and 85.2 % on WebShop, and Qwen3‑32B reaching 99.2 % on seen tasks and 94.0 % on unseen tasks.
By Qirui Liu, Wei Cao, Yu Mi, Yichen Sun, Xuekun Zhang, Lei Liu, Yue Shen, Zhixuan Chu
HINT-SD introduces a targeted self‑distillation framework for long‑horizon language‑model agents that uses full‑trajectory hindsight to identify failure‑relevant actions and applies feedback‑conditioned distillation only to those action spans. This selective approach reduces the need for per‑turn feedback, improving training efficiency and effectiveness. Experiments on BFCL v3 and AppWorld demonstrate that HINT‑SD outperforms dense per‑turn feedback baselines by up to 13.60 percentage points on average while cutting training time per step by 2.26×.
By Woongyeong Yeo, Yumin Choi, Taekyung Ki, Sung Ju Hwang
arXiv:2606. 11559v1 Announce Type: new Abstract: Reinforcement learning typically improves multi-turn agent capabilities through the terminal outcome of the trajectories, which makes it difficult to determine credit assignments for each intermediate turns.
By Haoran Liu, Yuwei Zhang, Xiyao Li, Bohan Lyu, Jingbo Shang
arXiv:2610.27847v1 Announce Type: new
Abstract: Scaling the training of Large Language Model (LLM) agents typically requires a large collection of executable environments, whose construction, mainten...
By Juncheng Diao, Haibin Wang, Wei Shi, Yifan Ding, Qingbin Li, Rongxiang Weng, Xunliang Cai
arXiv:2608. 04007v1 Announce Type: cross Abstract: Tool-Integrated Reasoning (TIR) enables LLMs to solve complex tasks through iterative tool interactions.
By Changle Qu, Sunhao Dai, Hengyi Cai, Yuqi Zhou, Xinran Chen, Simon, Jun Xu
The paper introduces SELF, a framework that jointly optimizes environmental feedback modeling and hindsight self-distillation for language agents in reinforcement learning settings where rewards are unavailable. SELF predicts environmental responses while distilling guidance from a feedback-conditioned self-teacher into the policy, creating a mutually reinforcing mechanism that strengthens both supervision and policy learning. Experiments with Qwen3-8B show that SELF outperforms SDPO and GRPO on $ au$-bench and AppWorld tasks, improving success rates and goal completion.
By Hangxi Guo, Fengyuan Liu, Yue Wang, Yuhua Qi, Haoyi Xiong, Fei Sun, Mengnan Du