arXiv Computation and Language By Omar Sharif, Eftekhar Hossain, Nikhil Singh, Patrick Ng

Disentangling Perception and Reasoning in Multimodal LLMs via Reward Design

Read the original on arXiv Computation and Language →

The paper investigates how reinforcement learning with verifiable rewards can improve multimodal large language models (LLMs) by separating perception and reasoning stages. Experiments on algorithmic visual puzzles show that perception, not reasoning, limits performance, and replacing images with textual descriptions boosts Claude models by over 20 points. Reward designs that encourage visual grounding during reasoning, such as GRPO for Qwen-2.5-VL-7B, yield structured reasoning and a 5.56‑point gain, though benefits vary across categories and trade‑offs exist between cross‑domain transfer and in‑domain accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computer Vision
Aug 25

VIG: Visual Information Gain as a Reward Signal for Multimodal Chain-of-Thought Compression

The paper introduces VIG (Visual Information Gain), an information‑theoretic reward that evaluates each token in a multimodal chain‑of‑thought by measuring how much the image reduces its predictive uncertainty. VIG is computed online using two forward passes—one with and one without the image—eliminating the need for reference chains or external annotations. Experiments on six multimodal reasoning benchmarks and multiple Qwen3‑VL‑Thinking model sizes show that VIG consistently improves the accuracy–efficiency trade‑off, demonstrating that efficient multimodal reasoning arises from increasing visual information density rather than merely limiting chain length.

By Wen Luo, Xiaohan Yi, Xiaotao Huang, Liqun Huang
arXiv Computer Vision
Oct 23

RECAP: Looking Once Is Not Enough for Vision-Language Reasoning

The paper introduces RECAP, a training method that enhances vision‑language models by incorporating temporally localized reactivation of image evidence—called visual revisit—into the reinforcement learning process. Visual revisit peaks are shown to predict correctness and causally improve performance; masking them hurts accuracy more than masking non‑peaks, while adding revisit peaks to failed reasoning partially restores correct answers. RECAP applies revisit‑conditioned advantage estimation, dynamic discounting, and gated value prediction to propagate final rewards through visually important reasoning steps, achieving significant gains on HallusionBench and other benchmarks without extra rewards or inference‑time changes.

By Zhaolu Kang, Tailong Luo, Chenxin Li, Yingjie He, Zhenyu Yu, Fengyu Zhou, Jiachen Qian, Lei Wei, Shuang Chen, Jiachen Li, Eric Hanchen Jiang, Rongchao Zhang, Zhengtao Yao, Hoi Leong Lee, Guansu Wang, Kaiyue Zhou
arXiv AI
Jun 12

PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment

arXiv:2603. 06652v2 Announce Type: replace-cross Abstract: Reinforcement learning has recently improved the reasoning ability of Large Language Models and Multimodal LLMs, yet prevailing reward designs emphasise final-answer correctness and consequently tolerate process hallucinations--cases where models reach the right answer while misperceiving visual evidence.

By Yantao Li, Qiang Hui, Chenyang Yan, Kanzhi Cheng, Fang Zhao, Chao Tan, Huanling Gao, Jianbing Zhang, Kai Wang, Xinyu Dai, Shiguo Lian