arXiv AI
Nov 5

GIST: Gibbs-Targeted Hierarchical Data Selection for Offline RLVR

The paper introduces GIST, a Gibbs‑Targeted Hierarchical Data Selection method for offline reinforcement learning with verifiable rewards (RLVR). By analyzing the bidirectional KL divergence between a policy and its Gibbs‑optimal target, GIST selectively supervises correct responses on‑policy and incorrect responses near the policy’s mode, while focusing token supervision on mismatched tokens. Experiments on GSM8K, MATH‑500, and MBPP show that GIST improves average accuracy across three models and outperforms seven baselines on Qwen3‑0.6B/GSM8K, all while supervising 66.6% fewer tokens.

By Xuliang Huang, Tianxiang Zhao, Chengwei Qin, Yao Shu
arXiv AI
Oct 31

SCARL: Sequence Confidence-Aligned Reinforcement Learning

SCARL (Sequence Confidence-Aligned Reinforcement Learning) addresses a misalignment between a reinforcement learning policy’s confidence and its reward signal in language-model reasoning. By reshaping each advantage with an alignment gap that reflects the difference between a response’s group-relative confidence and the outcome it certifies, SCARL drives responses with equal advantage toward equal confidence and mitigates entropy collapse. Experiments on Qwen3‑4B‑Base show a 2.4‑point lift in Pass@32 over DAPO and consistent improvements across multiple estimators and backbones while preserving policy entropy.

By Zhenghong Ke, Jingbo Zhou, Hua Wu
arXiv AI
Aug 11

Improving Generalization Robustness of Multimodal RLVR

arXiv:2608. 08802v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) makes Multimodal Large Language Models more accurate, but the gains are brittle: simply paraphrasing a question or changing the prompt template can degrade them, which challenges reliable deployment in high-stakes scenarios like medical VQA.

By Pengfei Zhou, Zhiwei Tang, Xiaopeng Peng, Chenrui Zhou, Lama Moukheiber, Yixing Ma, Bin Xu, Jiajun Song, Zhenglin Wan, Wangbo Zhao, Jiasheng Tang, Bohan Zhuang, Fan Wang, Yang You