arXiv Machine Learning June 25, 2026 By Wenyang Hu, Junxiang Jia, Zhen Shu, Daniel Dahlmeier, See-Kiong Ng, Bryan Kian Hsiang Low

ExTra: Exploratory Trajectory Optimization for Language Model Reinforcement Learning

Read the original on arXiv Machine Learning →

arXiv:2606. 24994v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) for language-model reasoning can fail at both extremes of task difficulty: easy prompts often produce all-correct, low-diversity rollout groups with little gradient signal, while hard prompts can produce all-incorrect groups with no positive reward.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Jun 3

GRAIL: Gradient-Reweighted Advantages for Reinforcement Learning with Verifiable Rewards

Reinforcement learning with verifiable rewards (e. g.

llms reinforcement-learning safety

arXiv AI

Jun 9

MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting

arXiv:2601. 09085v2 Announce Type: replace-cross Abstract: Group Relative Policy Optimization (GRPO) has become a standard approach for training mathematical reasoning models; however, its reliance on multiple completions per prompt makes training computationally expensive.

By Kangda Wei, Ruihong Huang

benchmarks

arXiv AI

Jul 21

It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branches

arXiv:2607. 16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrasting multiple self generated rollouts.

By Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Liwei Qian, Xin Pei, Jizhou Huang

llms reinforcement-learning benchmarks

arXiv AI

Jun 10

Reasoning or Memorization? Direction-Aware Diversity Exploration in LLM Reinforcement Learning

arXiv:2606. 10346v1 Announce Type: new Abstract: Reinforcement learning has become a key paradigm for eliciting reasoning abilities in large language models, where exploration is crucial for discovering effective solution trajectories.

By Jiangnan Xia, Yucheng Shi, Yu Yang, Kishan Panaganti, Zhenwen Liang, Ninghao Liu

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 16

A Gradient Perspective on RLVR Stability and Winner Advantage Policy Optimization

arXiv:2606. 16154v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves language-model reasoning, but GRPO-style optimization remains prone to collapse.

By Prasanth YSS, Zhichen Ren, Rasa Hosseinzadeh, Ilan Gofman, Yuqi Chen, Zhaoyan Liu, Guangwei Yu, Jesse C. Cresswell, Satya Krishna Gorti

reinforcement-learning benchmarks

arXiv AI

Jun 9

Momentum for Reasoning: Dense Intrinsic Signals in Policy Optimization

arXiv:2606. 08815v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for eliciting long-chain reasoning in large language models.

By Hao Chen, Zhanming Shen, Liyao Li, Yanyu Chen, Xuhang Zhu, Xiaomeng Hu, Qi Zhang, Ru Peng, Xiaoyu Shen, Haobo Wang, Junbo Zhao

llms reinforcement-learning benchmarks

Related stories

GRAIL: Gradient-Reweighted Advantages for Reinforcement Learning with Verifiable Rewards

MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting

It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branches

Reasoning or Memorization? Direction-Aware Diversity Exploration in LLM Reinforcement Learning

A Gradient Perspective on RLVR Stability and Winner Advantage Policy Optimization

Momentum for Reasoning: Dense Intrinsic Signals in Policy Optimization