TextualRL is a gradient‑free reinforcement learning framework that updates the textual context guiding large language model agents without altering their parameters. It critiques groups of rollouts from the same task, contrasts successful and failed runs, and uses cross‑group evidence to propose and validate context edits. Across six benchmarks, TextualRL boosts test accuracy by 24.4 and 15.3 percentage points for GPT‑5.5 and Qwen3.8‑27B, respectively.
ContextPilot is a proactive context‑management framework designed to improve long‑horizon agentic reasoning with large language models. It expands the toolset to include planning, long‑term memory, and soft context offloading, and introduces a reinforcement‑learning strategy that focuses on critical editing decisions and assigns action‑level advantages. Experiments on long‑context QA and deep search tasks demonstrate that ContextPilot achieves stronger performance with a more compact working context, outperforming existing baselines across various base models and benchmarks.
By Zhuoshi Pan, Qizhi Pei, Junru Lu, Honglin Lin, H. Vicky Zhao, Di Yin, Xing Sun
arXiv:2608. 05080v1 Announce Type: new Abstract: Critic-free group-based reinforcement learning has become a scalable approach for post-training large language models.
By Zheyuan Zhang, Manqing Mao, Hong Wang, Zhuoer Wang, Samson Koelle, Jie Yuan, Yanjun Lin, James Feng, Nikki Lijing Kuang, Yanfang Ye, Wei Niu
Matched-Context Group Policy Optimization (MCGPO) is a method for training long-horizon language model agents by constructing weighted action preferences from pre-collected trajectories. It defines comparison contexts using pre-action state summaries and rubric progress, and employs two-action support with exact and coarse matching rules to weight evidence for learning. Experiments across six environment-backbone settings show that MCGPO improves mean success rates over matched GraphGPO, achieving up to 5.16 percentage point gains on WebShop with Qwen3-4B and 8B models.
By Chenxu Zhang, Yaorui Shi, Wenlong Zhang, Jianchao Tan, Zihan Wang, Yiming Cheng
CAST is a critique‑aware training framework that transforms sparse task outcomes into action‑level supervision for both critique learning and policy optimization. By analyzing agent trajectories, CAST synthesizes structured rationales that explain action validity under partial observability, enabling the creation of richer training data. Fine‑tuned Qwen3‑family models trained with CAST show significant reliability gains, outperforming GPT‑OSS‑120B by over 10% on Retail tasks and improving Telehealth performance by 9% in an out‑of‑domain setting.
By Amir Saeidi, Zehua Zhang, Rishitosh Singh, Naman Ahuja, Vivek Gupta, Ali Payani, Gaowen Liu, Jayanth Srinivasa, Chitta Baral
arXiv:2607. 04242v1 Announce Type: new Abstract: Group-based reinforcement learning (RL) has become an effective paradigm for improving large language model agents on long-horizon interactive tasks.
By Mingxuan Fan, Peiyang Liu
The paper introduces Effect-Marginal Policy Optimization (EMPO), a method that addresses credit-update misalignment in multi-turn reinforcement learning for large language model agents. EMPO aligns policy updates with execution-level equivalence by using an effect-level surrogate that couples updates across different textual responses that produce the same effect. Experiments on ALFWorld, WebShop, and Sokoban demonstrate that EMPO improves both task performance and learning efficiency compared to strong baselines.
By Shaojin Ma, Ruitao Wang, Pengyi Li, Jie Yang, Liangyu Chen, Gangyi Zhao, Hongyao Tang, Yan Zheng, Jianye Hao
arXiv:2604. 00860v3 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a central post-training paradigm for improving the reasoning capabilities of large language models.
By Huaiyang Wang, Xiaojie Li, Deqing Wang, Haoyi Zhou, Zixuan Huang, Yaodong Yang, Jianxin Li, Yikun Ban
RubricEM is a unified reinforcement learning framework designed for deep research agents that plan, search, evaluate evidence, and synthesize long‑form reports. It introduces rubric‑guided policy decomposition, combining supervised fine‑tuning to scaffold agents with rubrics, stage‑structured GRPO for fine‑grained credit assignment, and a reflection meta‑policy for experience reuse. The resulting RubricEM‑8B model outperforms larger open models on four long‑form research benchmarks and approaches the performance of proprietary deep‑research systems.
By Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun-Liang Li, Long T. Le, Rujun Han, George Lee, Hanghang Tong, Chen-Yu Lee, Tomas Pfister
arXiv:2607. 21419v1 Announce Type: new Abstract: In long-horizon LLM agent reinforcement learning, weak policies often repeat similar failures, producing uninformative rollout trajectories and limiting effective policy optimization.
By Yipeng Shi, Zhipeng Ma, Yue Wang, Qitai Tan, Yang Li, Peng Chen, Zhengzhou Zhu
arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).
By Yixiu Mao, Yun Qu, Qi Wang, Heming Zou, Xiangyang Ji
arXiv:2607. 20668v1 Announce Type: cross Abstract: TextGrad improves language-model systems by revising text from feedback.
By Jaideep Ray, Ankit Goyal