MERIT is a three‑stage framework that enables language models to reason over table images by first adapting a table‑specific encoder through reconstruction, localization, and color‑conditioned supervision. It then fine‑tunes the model with supervised evidence selection and reasoning, and finally refines the policy using reinforcement learning with distinct rewards for evidence correctness, calculation, final answers, and output format. The approach is evaluated on table question answering and fact verification, demonstrating the benefits of visual inputs and training components across diverse table structures and visual presentations.
arXiv:2608.22429v1 Announce Type: new
Abstract: Multimodal Large Language Models (MLLMs) capable of thinking with images often rely on external tools for fine-grained perception. However, this relian...
By Changjiang Jiang, Qiannian Zhao, Lei Xin, Jinxiang Xie, Preslav Nakov, Zhuohan Xie
SLVR (Structured Latent Visual Reasoning) is a training framework that blends explicit chain-of-thought with latent reasoning for multimodal large language models. It organizes reasoning into typed latent stages—planning, grounding, evidence selection, and integration—each supervised by corresponding signals such as plans, bounding boxes, visual evidence, and final rationales. Built on Qwen2.5‑VL‑7B, SLVR consistently improves performance on multimodal reasoning benchmarks, achieving significant gains on MMVP, BLINK Relation, V*, MathVista, and ChartQA.
By Albert Gao, Bing Xue, Andrea Zanette
arXiv:2606. 17678v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images, indicating ineffective utilization of visual evidence during inference.
By Yilian Liu, Sicong Leng, Guoshun Nan, Junyi Zhu, Jiayu Huang, Minghao Sun, Xuancheng Zhu, Yisong Chen, Zexian Wei, Xiaofeng Tao
arXiv:2608.21796v1 Announce Type: cross
Abstract: Knowledge-based Visual Question Answering (KB-VQA) aims to answer queries that necessitate reasoning over external knowledge sources beyond the visua...
By Long Shu, Shuochen Liu, Wei Chen, Junda Lin, Zhi Zheng, Huijun Hou, Tong Xu
arXiv:2606. 29984v1 Announce Type: new Abstract: Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs).
By Peng, Lee, Yin Zhang, Yanglin Zhang, Haonan Wu, Zishan Liu, Ruoxi Zang, Xin Zhu, Jiayin Zheng, Jian Yao, Zefeng Ji, Fei Ma