arXiv Machine Learning October 22, 2026 By Yeonbum Kim, Hyun Kyu Lee, Sung Whan Yoon

Flat-RLVR: Sharpness-Aware Reinforcement Learning with Verifiable Rewards for Reasoning Capacity in LLMs

Read the original on arXiv Machine Learning →

Flat-RLVR introduces a sharpness-aware approach to reinforcement learning with verifiable rewards (RLVR) for large language models (LLMs). By targeting flat reward regions, the method reduces distribution sharpening, thereby preserving sample diversity while maintaining high single-attempt accuracy. The technique combines flat reward optimization with group-relative policy optimization (GRPO) variants to improve the trade-off between sampling efficiency and reasoning capacity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 2

RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning

arXiv:2606. 01281v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs).

By Yixiu Mao, Yun Qu, Qi Wang, Heming Zou, Xiangyang Ji

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 2

Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation

arXiv:2605. 21125v2 Announce Type: replace Abstract: Group Relative Policy Optimization (GRPO), a prominent algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improving the reasoning capabilities of large language models (LLMs).

By Xixiang He, Qiyao Sun, Ao Cheng, Xingming Li, Xuanyu Ji, Hailun Lu, Runke Huang, Qingyong Hu

llms reinforcement-learning benchmarks

arXiv Computation and Language

Aug 28

Boosting LLM Exploration via Weak-Model Guidance in RLVR

The paper introduces a method to enhance large language model (LLM) exploration in Reinforcement Learning with Verifiable Rewards (RLVR) by guiding the target model with partial reasoning trajectories from smaller, weaker language models. This weak-model guidance disrupts over‑confidence, preserves generative diversity, and mitigates entropy collapse without extra fine‑tuning or complex reward designs. Experiments on mathematical benchmarks show consistent improvements over vanilla RLVR, especially as the number of allowed attempts ($k$) increases, indicating broader reasoning coverage.

By Xingyu Shen, Huishuai Zhang, Peng Li, Yinchun Wang, Dongyan Zhao

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Aug 5

Don't Peek at the Answer: Outcome-Masked Group Relative Policy Optimization for Label-Free RLVR

arXiv:2608. 03119v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves LLM reasoning but typically relies on ground-truth (GT) answers, limiting scalability.

By Yongshi Ye, Liang Zhang, Yidong Chen, Xiaodong Shi, Biao Fu

llms reinforcement-learning benchmarks

arXiv AI

Jul 21

It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branches

arXiv:2607. 16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrasting multiple self generated rollouts.

By Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Liwei Qian, Xin Pei, Jizhou Huang

llms reinforcement-learning benchmarks

arXiv Machine Learning

Aug 18

Le Critique: Privileged Value Functions for LLM Reinforcement Learning

arXiv:2608. 16739v1 Announce Type: new Abstract: Reinforcement learning algorithms for Large Language Models (LLMs) are largely distinguished by their variance reduction strategy.

By Siddarth Venkatraman, Matthieu Dinot, Laurence Aitchison

llms reinforcement-learning efficiency

One email a morning, machine-written

Related stories

RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning

Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation

Boosting LLM Exploration via Weak-Model Guidance in RLVR

Don't Peek at the Answer: Outcome-Masked Group Relative Policy Optimization for Label-Free RLVR

It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branches

Le Critique: Privileged Value Functions for LLM Reinforcement Learning