arXiv Machine Learning

VRPRM: Process Reward Modeling via Visual Reasoning

arXiv:2508. 03556v4 Announce Type: replace Abstract: Process Reward Model (PRM) is widely used in the post-training of Large Language Model (LLM) because it can perform fine-grained evaluation of the reasoning steps of generated content.

arXiv AI

Jun 11

ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward

arXiv:2606. 11209v1 Announce Type: cross Abstract: Visual question answering increasingly requires multi-step reasoning.

By Jingpei Wu, Xiao Han, Weixiang Shen, Boer Zhang, Zifeng Ding, Volker Tresp

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computation and Language

4d ago

Question-Specific Knowledge Graphs for Efficient Visual Reasoning

arXiv:2609.35942v1 Announce Type: new Abstract: Recent work in visual question answering has shown that vision-language models can exhibit strong reasoning capabilities by translating visual inputs i...

By Ting-Chih Chen, Emile van Krieken, Shujian Yu, Filip Ilievski

llms nlp reinforcement-learning multimodal benchmarks

arXiv AI

6d ago

Stepwise Intrinsic Rewards for Reasoning in Large Language Models

The paper introduces Stepwise Marginal Information Gain (MIG), an intrinsic process reward that evaluates how each reasoning step of a large language model (LLM) or vision-language model (VLM) improves the likelihood of the reference answer. MIG rewards only new likelihood maxima, preventing duplicate credit, and is combined with outcome, format, and self‑distillation objectives to guide training. Experiments on eight benchmarks show that this method outperforms outcome‑only reinforcement learning and improves accuracy by up to 4.8 points over binary‑reward training, including a 12.6‑point gain on MathVerse and a 12.9‑point advantage on vision‑language transfer at 7B parameters.

By Xiangwei Wang, Wei Wang, Ken Chen, Nanduni Nimalsiri, Sachith Seneviratne, Saman Halgamuge

llms rag reinforcement-learning efficiency multimodal benchmarks

arXiv Machine Learning

Jun 9

The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning

arXiv:2606. 09078v1 Announce Type: new Abstract: Process Reward Models (PRMs) improve credit assignment for reasoning by providing step-level feedback.

By Aakriti Agrawal, Souradip Chakraborty, Armin Saghafian, Nihal Sharma, Rizal Fathony, Nam H Nguyen, C. Bayan Bruss, Amrit Singh Bedi, Furong Huang

safety

arXiv AI

3d ago

Video Understanding Reward Modeling: A Robust Benchmark and Performant Reward Models

arXiv:2605.07872v2 Announce Type: replace-cross Abstract: Multimodal reward models have advanced substantially in text and image domains, yet progress in video understanding reward modeling remains s...

By Yuancheng Wei, Linli Yao, Lei Li, Haojie Zhang, Hao Zhou, Fandong Meng, Xu Sun

reinforcement-learning multimodal benchmarks

arXiv AI

4d ago

Inducing Process Supervision from Outcome-Only Reinforcement Learning

arXiv:2609.36641v1 Announce Type: cross Abstract: Process reward models (PRMs) have become a key component for LLMs, as their step-level feedback supports both post-training and test-time reasoning....

By Shengda Fan, Xin Cong, Zhong Zhang, Haotian Chen, Yankai Lin

llms agents reinforcement-learning benchmarks

arXiv AI

Sep 1

ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth

arXiv:2512.03244v2 Announce Type: replace-cross Abstract: Training process reward models (PRMs) requires step-level correctness labels, obtained either through expensive human annotation or by relyin...

By Salman Rahman, Sruthi Gorantla, Arpit Gupta, Swastik Roy, Nanyun Peng, Yang Liu

llms reinforcement-learning benchmarks

arXiv AI

Jul 21

It Takes 8 Tokens: Weak-to-Strong Off-Policy RL via Auxiliary Branches

arXiv:2607. 16205v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards has emerged as a standard approach for enhancing reasoning in large language models, which typically optimizes the policy by contrasting multiple self generated rollouts.

By Dayu Wang, Jiaye Yang, Weikang Li, Jiahui Liang, Liwei Qian, Xin Pei, Jizhou Huang

llms reinforcement-learning benchmarks

arXiv AI

Jun 26

Joint Reward Modeling: Internalizing Chain-of-Thought for Efficient Visual Reward Models

arXiv:2602. 07533v2 Announce Type: replace Abstract: Reward models are critical for reinforcement learning from human feedback, as they determine the alignment quality and reliability of generative models.

By Yankai Yang, Yancheng Long, Hongyang Wei, Wei Chen, Tianke Zhang, Kaiyu Jiang, Haonan Fan, Changyi Liu, Jiankang Chen, Kaiyu Tang, Bin Wen, Fan Yang, Tingting Gao, Han Li, Shuo Yang

reinforcement-learning multimodal benchmarks safety

arXiv AI

Sep 1

Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training

The paper investigates how different forms of compressed chain‑of‑thought (CoT) reasoning—Explicit, Composed, and Implicit—affect large language model (LLM) performance after supervised fine‑tuning (SFT). Using a synthetic compositional reasoning task, the authors show that coarser CoT requires more SFT data, that Composed and Implicit CoT benefit more from data scaling (with Composed also benefiting from repetition), and that reinforcement learning with verifiable rewards (RLVR) can decompose compressed steps learned during SFT. Additionally, unidirectional CoT ordering improves generalization on longer sequential tasks.

By Kohsei Matsutani, Gouki Minegishi, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo

llms reinforcement-learning fine-tuning

arXiv Computation and Language

Sep 3

On Asymmetric Optimization of Reasoning and Perception in Vision-Language Model Post-Training

The paper investigates why post‑training boosts reasoning more than perception in vision‑language models. Using a diagnostic framework with synthetic tasks, it finds a perception‑reasoning asymmetry: supervised fine‑tuning suffers from token imbalance, while reinforcement learning suffers from reward coupling. The authors propose reweighting losses and perception‑aware rewards, achieving up to 18.2‑point and 6.0‑point gains respectively, and show that these methods also improve real‑world visual reasoning by up to 3.3 points.

By Xueqing Wu, Yu-Chi Lin, Kai-Wei Chang, Nanyun Peng

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jun 19

Vero: An Open RL Recipe for General Visual Reasoning

arXiv:2604. 04917v3 Announce Type: replace-cross Abstract: What does it take to build a visual reasoner that works across charts, science, spatial understanding, and open-ended tasks?

By Gabriel Sarch, Linrong Cai, Qunzhong Wang, Haoyang Wu, Danqi Chen, Zhuang Liu

llms reinforcement-learning efficiency multimodal benchmarks

Related stories