arXiv Computer Vision
Sep 18

Region-Level Policy Optimization for Fine-grained MLLM Perception

The paper introduces Vision‑RL2, a region‑level reinforcement learning approach that optimizes a lightweight proposal network for fine‑grained multimodal large language model (MLLM) perception. By treating coherent image regions as actions and scoring them with a frozen MLLM reader, the method selectively focuses visual resolution on evidence, reducing token usage while improving accuracy across multiple benchmarks and backbones. The approach eliminates the need for region annotations, response sampling, or reasoning trajectories, and the refined proposals enable sparse encoding that magnifies relevant evidence.

By Yuheng Shi, Xiaohuan Pei, Minjing Dong, Chang Xu
arXiv AI
6d ago

ZoomGain: Mining Global-to-Zoom Perception Gaps for Fine-Grained Visual Supervision

ZoomGain introduces a data construction pipeline that exploits the global-to-zoom perception gap to generate fine‑grained visual questions. By creating questions from full‑image context and verifying answers only on zoomed‑in views, the method ensures unambiguous, language‑prior‑free queries and high‑quality distractors for multiple‑choice tasks. Using this pipeline, the authors built ZoomGainBench, a 1,000‑question bilingual benchmark, and demonstrated that training on 9 K ZoomGain samples markedly boosts the visual perception abilities of 4B/9B VLMs, surpassing many larger models on perception benchmarks.

By Haozhan Shen, Liangbo He, Jun Lan, Huijia Zhu, Weiqiang Wang, Tiancheng Zhao, Jianwei Yin