TextualRL is a gradient‑free reinforcement learning framework that updates the textual context guiding large language model agents without altering their parameters. It treats groups of rollouts from the same task as units of critique, contrasting successful and failed runs to generate context patches, and validates these patches using held‑out rewards. Experiments on six benchmarks show that TextualRL boosts test accuracy by 24.4 and 15.3 percentage points for GPT‑5.5 and Qwen3.8‑27B, respectively.
By Tianyu Huai, Jie Zhou, Bo Mao, Qin Chen, Liang He, Xipeng Qiu
ContextPilot is a proactive context‑management framework designed to improve long‑horizon agentic reasoning with large language models. It expands the toolset to include planning, long‑term memory, and soft context offloading, and introduces a reinforcement‑learning strategy that focuses on critical editing decisions and assigns action‑level advantages. Experiments on long‑context QA and deep search tasks demonstrate that ContextPilot achieves stronger performance with a more compact working context, outperforming existing baselines across various base models and benchmarks.
By Zhuoshi Pan, Qizhi Pei, Junru Lu, Honglin Lin, H. Vicky Zhao, Di Yin, Xing Sun
arXiv:2608. 05080v1 Announce Type: new Abstract: Critic-free group-based reinforcement learning has become a scalable approach for post-training large language models.
By Zheyuan Zhang, Manqing Mao, Hong Wang, Zhuoer Wang, Samson Koelle, Jie Yuan, Yanjun Lin, James Feng, Nikki Lijing Kuang, Yanfang Ye, Wei Niu
arXiv:2607. 20668v1 Announce Type: cross Abstract: TextGrad improves language-model systems by revising text from feedback.
By Jaideep Ray, Ankit Goyal
The paper introduces Effect-Marginal Policy Optimization (EMPO), a method that addresses credit-update misalignment in multi-turn reinforcement learning for large language model agents. EMPO aligns policy updates with execution-level equivalence by using an effect-level surrogate that couples updates across different textual responses that produce the same effect. Experiments on ALFWorld, WebShop, and Sokoban demonstrate that EMPO improves both task performance and learning efficiency compared to strong baselines.
By Shaojin Ma, Ruitao Wang, Pengyi Li, Jie Yang, Liangyu Chen, Gangyi Zhao, Hongyao Tang, Yan Zheng, Jianye Hao
RubricEM is a unified reinforcement learning framework designed for deep research agents that plan, search, evaluate evidence, and synthesize long‑form reports. It introduces rubric‑guided policy decomposition, combining supervised fine‑tuning to scaffold agents with rubrics, stage‑structured GRPO for fine‑grained credit assignment, and a reflection meta‑policy for experience reuse. The resulting RubricEM‑8B model outperforms larger open models on four long‑form research benchmarks and approaches the performance of proprietary deep‑research systems.
By Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun-Liang Li, Long T. Le, Rujun Han, George Lee, Hanghang Tong, Chen-Yu Lee, Tomas Pfister