arXiv Computation and Language

Question-Specific Knowledge Graphs for Efficient Visual Reasoning

arXiv Computation and Language
Nov 2

GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning

The paper introduces GRIT, a training framework that adds visual grounding to reinforcement learning with verifiable rewards (RLVR) for vision-language models, addressing the issue of Correct-but-Visually-Misgrounded Reasoning (CVMR). CVMR occurs when models produce correct final answers while misperceiving visual information during intermediate reasoning steps, a problem that standard RLVR rewards cannot detect. By analyzing token-level attention patterns, the authors show that CVMR tokens differ from well-grounded tokens and use this insight to guide GRIT, which improves both reasoning accuracy and grounding fidelity. Additionally, the authors present GRIT-Bench, a new benchmark for evaluating text-rich reasoning across varying text lengths and reasoning depths.

By Chanseul Cho, Noori Bae, Dongheon Lee, Youngmin Ro
arXiv Computer Vision
Oct 23

RECAP: Looking Once Is Not Enough for Vision-Language Reasoning

The paper introduces RECAP, a training method that enhances vision‑language models by incorporating temporally localized reactivation of image evidence—called visual revisit—into the reinforcement learning process. Visual revisit peaks are shown to predict correctness and causally improve performance; masking them hurts accuracy more than masking non‑peaks, while adding revisit peaks to failed reasoning partially restores correct answers. RECAP applies revisit‑conditioned advantage estimation, dynamic discounting, and gated value prediction to propagate final rewards through visually important reasoning steps, achieving significant gains on HallusionBench and other benchmarks without extra rewards or inference‑time changes.

By Zhaolu Kang, Tailong Luo, Chenxin Li, Yingjie He, Zhenyu Yu, Fengyu Zhou, Jiachen Qian, Lei Wei, Shuang Chen, Jiachen Li, Eric Hanchen Jiang, Rongchao Zhang, Zhengtao Yao, Hoi Leong Lee, Guansu Wang, Kaiyue Zhou
arXiv Machine Learning
Jul 7

Incentivizing Vision Language Models to Search for Long Video Question Answering

arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.

By Harsh Goel, S P Sharan, Sahil Shah, Minkyu Choi, Joungbin An, Kristen Grauman, Sandeep P. Chinchali
arXiv AI
Sep 17

Anchoring What Matters: A Dual-Level Learning Framework for Visually-Grounded Multimodal Reasoning

The paper introduces PIVOT, a dual-level learning framework designed to improve visually-grounded multimodal reasoning in large vision-language models. PIVOT employs a self‑calibrated experience replay mechanism to selectively reuse valuable visual reasoning trajectories, and a vision‑guided advantage allocation scheme that assigns extra rewards to tokens with strong visual support. Experiments on multiple benchmarks show that PIVOT enhances the multimodal reasoning performance of these models.

By Xinxin Song, Siyuan Li, Tingxiong Xiao, Jinli Suo
arXiv AI
Oct 9

SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows

SLVR (Structured Latent Visual Reasoning) is a training framework that blends explicit chain-of-thought with latent reasoning for multimodal large language models. It organizes reasoning into typed latent stages—planning, grounding, evidence selection, and integration—each supervised by corresponding signals such as plans, bounding boxes, visual evidence, and final rationales. Built on Qwen2.5‑VL‑7B, SLVR consistently improves performance on multimodal reasoning benchmarks, achieving significant gains on MMVP, BLINK Relation, V*, MathVista, and ChartQA.

By Albert Gao, Bing Xue, Andrea Zanette
arXiv AI
23h ago

Rethinking Visual Reasoning in Text-to-Image Reward Modeling

The paper examines how current reward models used in text-to-image systems struggle with visual reasoning tasks. By introducing a data‑centric approach that flips the pairing axis—using visual reasoning datasets as preference pairs with the same image but different prompts—the authors achieve simultaneous gains in visual understanding and preference accuracy. This method yields measurable improvements in generation quality on compositional prompts, outperforming a larger baseline model.

By Shiye Su, Xiaohan Wang, Ludwig Schmidt, Serena Yeung-Levy
arXiv AI
Jun 17

MathVis-Fine: Aligning Visual Supervision with Necessity via Progressive Dependency-Guided Training for Multimodal Mathematical Reasoning

arXiv:2606. 17888v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning has extended from purely linguistic domains to multimodal scenarios; however, existing approaches often treat visual inputs as homogeneous or auxiliary signals, failing to capture the intricate and sample-specific dependencies between text and images in mathematical problem-solving.

By Wanshi Xu, Haokun Zhao, Haidong Yuan, Songjun Cao, Long Ma
arXiv AI
Sep 15

UniCAR-RL: Seeing Better before Thinking Deeper in Visual Mathematics

UniCAR‑RL is an annotation‑free reinforcement learning framework designed to improve multimodal large language models’ visual mathematics reasoning. It decouples perception and reasoning by using three branches: Caption‑RL for perception optimization, Reasoning‑RL for logical reasoning with a gold image description, and QA‑RL for end‑to‑end question answering. Experiments show significant gains in mathematical and visual reasoning across various model architectures and scales using only raw short‑answer data.

By Yuzhe Li, Hao Yan, Hao Wang, Xingchen Liu, Ya-Qi Yu, Jihao Wu, Minghui Liao, Wei Chen, Yuliang Liu