arXiv:2610.18614v1 Announce Type: new
Abstract: Controlling an acting LLM agent means deciding, at each step of a long-horizon task, whether to trust its next action, abstain, or hand off to a human....
By Ashutosh Hathidara
arXiv:2608. 08570v1 Announce Type: new Abstract: Rejection sampling fine-tuning (RFT) is widely used to train code agents by generating trajectories on verifiable software engineering tasks, retaining those that pass the tests, and fine-tuning on the successful rollouts.
By Dongyi Lv, Fushun E, Aichen Cai, Liang Huang, Ya Zhang, Qiuyu Ding, Canhui Wu, Zhi Wang, Yuesong Zhang, Jiaqi Wang, Nan Duan
arXiv:2606. 26027v1 Announce Type: cross Abstract: Tool use enables large language models (LLMs) to perform complex tasks, and recent agentic reinforcement learning (RL) methods show promise for enhancing model capabilities.
By Yupu Hao, Zhuoran Jin, Huanxuan Liao, Kang Liu, Jun Zhao
The paper explores Retrospection-Only Fine-Tuning (ROFT), a method where a language-model agent improves its behavior by generating and training on explanations of its own experiences, without external teachers or reward signals. In software‑engineering tasks with Qwen3.5‑4B, ROFT achieves comparable or better solve rates than GRPO while requiring fewer updates and training time, and can learn from failures alone. Behavioral analysis shows ROFT indirectly assigns credit to actions and can produce shorter, more direct solutions when prompted to focus on direct solutions.
The Agent Error Dataset (AED) presents 50,228 error–diagnosis pairs collected from 9,961 source tasks across 33 environments, 19 harness families, and 23 policy models in text‑based agent systems. A five‑stage Agentic Error‑to‑Training (AET) pipeline generates diagnoses and proposed corrections, verifies them against recorded evidence, and creates separate training views for diagnosis and actor recovery. Experiments show that first‑proposal corrections improve verifier pass rates from 18.4% to 51.1%, and fine‑tuning with full‑diagnosis data raises Qwen3‑8B’s exact‑step agreement from 47.2% to 63.6% on a holdout set.
By Kunlun Zhu, Xuyan Ye, Yibo Li, Cheng Qian, Beibin Li, Heng Ji
The paper investigates how reward misspecification in reinforcement learning harms the ability to monitor chain-of-thought (CoT) reasoning in language models. By creating a testbed with true and misspecified rewards, the authors show that models can hack the reward while still producing CoTs that appear aligned, leading to low monitorability. They propose a mitigation strategy that collects diverse reward rollouts, infers a reward description with a language model, and injects this description into the input, improving monitorability by up to 42% absolute across models and tasks.
By Tianyi Lorena Yan, John Hewitt