arXiv Machine Learning

Noisy Data is Destructive to Reinforcement Learning with Verifiable Rewards

arXiv:2603. 16140v2 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) has driven recent capability advances of large language models across various domains.

arXiv Machine Learning
Jun 4

Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning

arXiv:2603. 09803v2 Announce Type: replace Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally, potentially reinforcing flawed traces that arrive at correct answers by chance.

By Tiehua Mei, Minxuan Lv, Leiyu Pan, Zhenpeng Su, Hongru Hou, Hengrui Chen, Ao Xu, Deqing Yang
arXiv Machine Learning
Sep 4

DE-Venus: A Data-Efficient RLVR Framework for Large Language Models

DE‑Venus is a unified, data‑efficient framework for reinforcement learning with verifiable rewards (RLVR) tailored to large language models. It structures the RLVR lifecycle into three modules—Active Data Selection, Weak Supervision Construction, and Training‑Time Supervision Refinement—allowing method‑specific decisions to be expressed as dataset transitions or online transformations while maintaining distributed execution contracts. Experiments on public benchmarks and three business scenarios show that DE‑Venus can preserve or improve model quality using only 10% of labels or 13% of relevant data, and can cut convergence steps by 63%–75% in selected business configurations.

By Shenzhi Yang, Guangcheng Zhu, Kai Tang, Zhengqing Zang, Xing Zheng, Haobo Wang, Yingfan Ma, Bowen Song, Bo Han, Bo An, Lei Feng, Weiqiang Wang, Junbo Zhao, Gang Chen
arXiv Machine Learning
Jun 9

RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments

arXiv:2511. 07317v2 Announce Type: replace-cross Abstract: We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scale up RL for language models (LMs).

By Zhiyuan Zeng, Hamish Ivison, Yiping Wang, Lifan Yuan, Shuyue Stella Li, Zhuorui Ye, Siting Li, Jacqueline He, Runlong Zhou, Tong Chen, Chenyang Zhao, Yulia Tsvetkov, Simon Shaolei Du, Natasha Jaques, Hao Peng, Pang Wei Koh, Hannaneh Hajishirzi
arXiv Machine Learning
Aug 12

Reinforcement Learning-based Semi-supervised Knowledge Distillation with LLM-as-a-Judge

arXiv:2604. 02621v2 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) substantially improves the reasoning capabilities of language models, but most existing RL fine-tuning approaches rely entirely on ground-truth verifiable rewards and thus labeled datasets with verifiable answers.

By Yiyang Shen, Lifu Tu, Weiran Wang
arXiv Machine Learning
Jul 21

Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR

arXiv:2509. 02522v3 Announce Type: replace-cross Abstract: Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have empowered large language models (LLMs) to tackle challenging reasoning tasks such as mathematics and programming, however existing RLVR methods often suffer from sparse reward signals and unstable policy gradient updates inherent to RL-based approaches.

By Jiaming Li, Longze Chen, Ze Gong, Yukun Chen, Lu Wang, Wanwei He, Run Luo, Min Yang
arXiv Machine Learning
Jun 9

ConSteer-RL: Steering Reasoning Capabilities in Large Language Models via Confidence-Aware Reinforcement Learning

arXiv:2606. 08088v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has recently become a key paradigm for improving the reasoning abilities of Large Language Models (LLMs), yet it remains limited by sparse binary rewards and its ignorance of model-internal uncertainty.

By Qing Miao, Yiming Zhao, Jing Yang, Chenxi Liu, Yuehai Chen, Yuewen Liu, Shaoyi Du, Badong Chen
arXiv AI
4d ago

Learning to Prove, Not Just to Answer: Reinforcement Learning from Formal Verification for Natural-Language Logical Reasoning

The paper introduces Proof‑R1, a reinforcement‑learning framework that trains large language models to generate verifiable proofs for natural‑language logical reasoning tasks. Proof‑R1 only accepts a generated conclusion into the proof state when it satisfies formal verification constraints, ensuring each reasoning step is machine‑checkable. The method also reconstructs the dependency closure that supports the final answer, aligning credit with valid proof steps, and shows improved answer accuracy and verifiability across multiple benchmarks and models.

By Qili Zhang, Qianren Mao, Hanze Cai, Kaiming Zhao, Yuening He, Xihan Lei, Yashuo Luo, Hanwen Hao, Yutong Gu, Likang Xiao, Zhijun Chen, Weifeng Jiang, Haoyi Zhou, Jianxin Li
arXiv AI
Aug 11

Improving Generalization Robustness of Multimodal RLVR

arXiv:2608. 08802v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) makes Multimodal Large Language Models more accurate, but the gains are brittle: simply paraphrasing a question or changing the prompt template can degrade them, which challenges reliable deployment in high-stakes scenarios like medical VQA.

By Pengfei Zhou, Zhiwei Tang, Xiaopeng Peng, Chenrui Zhou, Lama Moukheiber, Yixing Ma, Bin Xu, Jiajun Song, Zhenglin Wan, Wangbo Zhao, Jiasheng Tang, Bohan Zhuang, Fan Wang, Yang You