Learned Reporting Preferences in RLVR Can Conflict with the Current Request
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The paper introduces GIST, a Gibbs‑Targeted Hierarchical Data Selection method for offline reinforcement learning with verifiable rewards (RLVR). By analyzing the bidirectional KL divergence between a policy and its Gibbs‑optimal target, GIST selectively supervises correct responses on‑policy and incorrect responses near the policy’s mode, while focusing token supervision on mismatched tokens. Experiments on GSM8K, MATH‑500, and MBPP show that GIST improves average accuracy across three models and outperforms seven baselines on Qwen3‑0.6B/GSM8K, all while supervising 66.6% fewer tokens.
SCARL (Sequence Confidence-Aligned Reinforcement Learning) addresses a misalignment between a reinforcement learning policy’s confidence and its reward signal in language-model reasoning. By reshaping each advantage with an alignment gap that reflects the difference between a response’s group-relative confidence and the outcome it certifies, SCARL drives responses with equal advantage toward equal confidence and mitigates entropy collapse. Experiments on Qwen3‑4B‑Base show a 2.4‑point lift in Pass@32 over DAPO and consistent improvements across multiple estimators and backbones while preserving policy entropy.
arXiv:2507.14843v5 Announce Type: replace Abstract: Recent advances highlight Reinforcement Learning with Verifiable Rewards (RLVR) as a promising approach to enhancing LLM capabilities. However, it...
arXiv:2605. 02909v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has become a powerful approach for improving the reasoning capabilities of large language models (LLMs).
arXiv:2609.36572v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has been extended to Large Vision-Language Models (LVLMs), and perception-aware methods further e...
arXiv:2608. 08802v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) makes Multimodal Large Language Models more accurate, but the gains are brittle: simply paraphrasing a question or changing the prompt template can degrade them, which challenges reliable deployment in high-stakes scenarios like medical VQA.