Matched-Context Group Policy Optimization (MCGPO) is a method for training long-horizon language model agents by constructing weighted action preferences from pre-collected trajectories. It defines comparison contexts using pre-action state summaries and rubric progress, and employs two-action support with exact and coarse matching rules to weight evidence for learning. Experiments across six environment-backbone settings show that MCGPO improves mean success rates over matched GraphGPO, achieving up to 5.16 percentage point gains on WebShop with Qwen3-4B and 8B models.
By Chenxu Zhang, Yaorui Shi, Wenlong Zhang, Jianchao Tan, Zihan Wang, Yiming Cheng
arXiv:2606. 05885v1 Announce Type: new Abstract: Long-horizon LLM agents require reinforcement learning methods that can assign credit to intermediate decisions under sparse and delayed rewards.
By Yuanfan Li, Qi Zhou, Wenjing Duan, Lu Chen
arXiv:2607. 16244v1 Announce Type: cross Abstract: Training multi-turn evidence-reading agents with outcome-only reinforcement learning is unstable because intermediate turns receive little direct credit.
By Hao Dou
arXiv:2610.19789v1 Announce Type: cross
Abstract: Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published bas...
By Linhao Zhong, Bozhen Fang, Xuehao Wang, Duochao Shi, Zekai Luo, Guangming Yao, Yi Yuan, Hao Chen, Jingdong Chen, Chunhua Shen
arXiv:2610.28072v1 Announce Type: new
Abstract: Learning rate schedules are typically fixed before training begins, yet when fine-tuning large language models with low-rank adapters (LoRA), the appro...
By Jan Kobiolka, Christian Frey, Josif Grabocka
Agentic ESOpt proposes using evolution strategies (ES) instead of reinforcement learning to fine‑tune large language‑model agents for long‑horizon tasks. ES offers model scalability, flexibility, and better long‑horizon credit assignment, enabling full‑parameter optimization with minimal GPU memory. The framework samples parameter perturbations, evaluates agents with rewards, and updates online, achieving notable performance gains on WebArena‑Lite and in test‑time prompt‑parameter co‑evolution.
By Zhi Zheng, Rongsheng Chen, Yunpeng Ba, Zhenkun Wang, Yee Whye Teh, Wee Sun Lee
Gibbs‑Gap Policy Optimization (GGPO) is a new off‑policy reinforcement learning method that avoids high‑variance learner‑to‑behavior importance weights by minimizing a log‑mean‑exp contrast of KL‑adjusted rewards across responses to the same prompt. Derived from KL‑regularized policy optimization, GGPO guarantees that a learner’s single‑response expected reward is at least the behavior policy’s best‑of‑K reward minus bounded terms, and it includes a reference‑refresh mechanism that empirically improves performance. Experiments on benchmark problems and mathematical reasoning tasks show GGPO outperforming GRPO and AGRO, especially under training‑inference mismatch and with fixed offline data.
whyItMatters":"The paper demonstrates a practical improvement in off‑policy RL for language models by reducing variance and ensuring better reward guarantees, which is directly supported by the reported empirical gains and theoretical bounds."
By Xuanchang Zhang, Tong Zhang
The paper introduces GDEPO, a reinforcement learning framework designed to improve sample efficiency in automated theorem proving. GDEPO incorporates dynamic additional sampling, an equal‑right advantage function that separates correctness from reward magnitude, and dynamic additional iterations to refine learning from initially failed samples. Experiments on MinF2F‑test, MathOlympiadBench, and PutnamBench demonstrate that these mechanisms synergistically enhance data utilization and optimization efficiency.
By Zhengqing Yan, Fan Guo, Xinyang Liu, Yi Zhang, Yao Liu, Qi Liu, Jihao Huang, Kang Song
arXiv:2607. 27787v1 Announce Type: new Abstract: Reinforcement learning from verifiable rewards (RLVR) for mathematical reasoning suffers from a structural blind spot: on "cliff" prompts-those on which every sampled rollout in a group fails-the group-normalized advantage is identically zero, so GRPO produces no gradient on precisely the prompts at the frontier of the model's capability.
By Ken Ding
KnowGPO introduces a reinforcement learning framework that groups search agent trajectories by the task-relevant knowledge they acquire, rather than by surface observations. By decomposing required information into atomic knowledge conditions and using the incremental knowledge gain as a process reward, KnowGPO provides finer-grained credit assignment for deep search tasks. Experiments on Xbench, BrowseComp-ZH, and GAIA demonstrate that KnowGPO outperforms outcome-only and process-supervision baselines by 3.8 to 4.2 points, with an 8B model rivaling or exceeding most 32B open-source systems.
By Siqi Chen, Zhengxi Lu, Yuchen Yan, Xingyu Wu, Qipeng Chen, Xin Zhang, Aiting Liu, Chao Deng, Jie Liu, Jin Ma, Weiming Lu, Jun Xiao, Yongliang Shen
Reinforcement learning from verifiable rewards (RLVR) for mathematical reasoning suffers from a structural blind spot: on "cliff" prompts-those on which every sampled rollout in a group fails-the group-normalized advantage is identically zero, so GRPO produces no gradient on precisely the prompts at the frontier of the model's capability. We introduce LoRA Scaffolded Policy Optimization (LSPO), a sampling-time mechanism that recovers this lost gradient.
arXiv:2607. 09375v1 Announce Type: new Abstract: We present Mach-Mind-4-Flash, a 35B-parameter Mixture-of-Experts (MoE) agentic model with 3B activated parameters.
By Foundation Model Team