arXiv AI October 15, 2026 By Chaoli Mou, Xinning Chen, Zhan Zhuang, Qinghe Zheng, Deyuan Kong, Yu Zhang

Selective Successor Eligibility Traces for RLVR

The paper introduces Selective Successor Eligibility Traces (S-trace), a critic‑free eligibility trace method designed to improve temporal credit assignment in Reinforcement Learning with Verifiable Rewards (RLVR). S-trace reweights successor importance to recover downstream credit signals under PPO-style clipping and selectively propagates them to historical tokens using entropy-based token selection. Experiments on Qwen3 models show that S-trace yields significant absolute gains in average pass@16 over GRPO and trace-style GRPO, while also producing shorter responses.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 16

A Gradient Perspective on RLVR Stability and Winner Advantage Policy Optimization

arXiv:2606. 16154v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves language-model reasoning, but GRPO-style optimization remains prone to collapse.

By Prasanth YSS, Zhichen Ren, Rasa Hosseinzadeh, Ilan Gofman, Yuqi Chen, Zhaoyan Liu, Guangwei Yu, Jesse C. Cresswell, Satya Krishna Gorti

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Learning with a Single Rollout via Monte Carlo Pass@k Critic

arXiv:2606. 25451v1 Announce Type: new Abstract: Estimating token-level advantages in reinforcement learning (RL) for language models remains challenging because scaling up episodic experience collection is expensive.

By Fengdi Che, Yang Liu, Lei Yu, Meng Cao, Tong Che, Rupam Mahmood, Dale Schuurmans

llms reinforcement-learning benchmarks

arXiv Machine Learning

1d ago

Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling

arXiv:2610.20497v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) trains language models from checked task outcomes, but assigning a terminal reward to individual...

By Wei Chen, Yubing Wu, Zhou Xu, Haoyue Zhang, Delu Zeng, Jiahao Wang, Junmei Yang, Pan tiewen, Jingfeng Liu, Zhiheng Zhou, Bo Li, Min Chen

llms reinforcement-learning

arXiv AI

Sep 15

HISPO: Hierarchical Importance-Sampling Policy Optimization with Entropy-Derived Segments

HISPO introduces a segment‑level policy‑optimization technique for reinforcement learning with verifiable rewards, creating entropy‑derived contiguous segments during rollout and applying clipped importance‑sampling correction at this granularity. It bridges the gap between token‑level and sequence‑level corrections, offering a middle‑ground approach for credit assignment in long‑form mathematical reasoning. Evaluated on Qwen3‑1.7B‑Base across six benchmarks, HISPO consistently outperforms or matches the strongest baselines in Pass@8 and Acc@8 metrics, notably improving AIME25 scores over GRPO and GSPO.

By Quoc-Vinh Lai-Dang, Hyo-Sang Shin

llms reinforcement-learning fine-tuning benchmarks

arXiv Computation and Language

Sep 22

What are Key Factors for Updates in RL for LLM Reasoning?

arXiv:2606.22570v2 Announce Type: replace Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a promising framework for enhancing the reasoning ability of large language mo...

By Peidong Wang, Demi Wang, Xufang Luo, Jiahang Xu, Xiaocui Yang, Shi Feng, Yuqing Yang, Dongsheng Li

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 19

VIMPO: Value-Implicit Policy Optimization for LLMs

arXiv:2606. 20008v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has become a central tool for improving the reasoning ability of large language models, but current methods face a trade-off between simplicity and credit assignment.

By Zhewei Kang, Aosong Feng, Sergey Levine, Dawn Song, Xuandong Zhao

llms reinforcement-learning benchmarks

One email a morning, machine-written

Related stories

A Gradient Perspective on RLVR Stability and Winner Advantage Policy Optimization

Learning with a Single Rollout via Monte Carlo Pass@k Critic

Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling

HISPO: Hierarchical Importance-Sampling Policy Optimization with Entropy-Derived Segments

What are Key Factors for Updates in RL for LLM Reasoning?

VIMPO: Value-Implicit Policy Optimization for LLMs