DocPilot is an agentic framework for long visual document understanding that orchestrates evidence seeking through page-level navigation and fine-grained localization, integrating visual inspection, code-based numerical reasoning, and cross-modal reasoning. It employs curriculum-based agentic training to enable small language models to learn complex document solving, progressing from single-ability learning to adaptive orchestration. Experiments on benchmarks such as MMLongBench-Doc, DocBench, FinRAGBench-V, and MADQA show that DocPilot achieves strong gains with frontier VLM backbones, and its 4B variant competes with much larger models, demonstrating effectiveness and generalization across diverse long-document reasoning tasks.
By Zhuowen Liang, Xiaotian Lin, Yuyu Luo, Haixun Wang, Nan Tang
arXiv:2608. 03292v1 Announce Type: new Abstract: Long Document Visual Question Answering (LongDocVQA) requires Multimodal Large Language Models (MLLMs) to locate, integrate, and reason over heterogeneous document elements distributed across multiple pages.
By Le Xiang, Zhicheng Guan, Hong Chen, Xiaocong Lin, Zhenghua Lei, Teng Hu, Bolei He, Long Zeng
arXiv:2511. 17731v2 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) prompting has proven remarkably effective for eliciting complex reasoning in large language models (LLMs).
By Lingxiao Li, Yifan Wang, Xinyan Gao, Chen Tang, Xiangyu Yue, Chenyu You
arXiv:2608.21431v1 Announce Type: new
Abstract: Knowledge-based Visual Question Answering aims to answer questions about an image by integrating external knowledge with visual and textual information...
By Qiyou Liu, Yong Zhang, Jianjie Luo, Zhenguo Yang, Yi Yu
arXiv:2609.05518v1 Announce Type: cross
Abstract: Despite the strong capabilities of multimodal large language models (MLLMs), their parametric knowledge remains incomplete and difficult to update, m...
By Jiacheng Cai, Zijin Hong, Zheng Yuan, Huachi Zhou, Qinggang Zhang, Xiao Huang
CoVA‑SFT is a new large‑scale dataset comprising 51.9K samples and over 222K multimodal reasoning steps that teach models to interleave text and visual abstractions across five layout families and 17 complex tasks. It includes explicit rationale formulations, agentic renderings, and verification loops to help models build and maintain internal visual workspaces for purely textual reasoning problems. A companion benchmark, CoVA‑Bench, contains 1,700 held‑out test samples for reproducible evaluation, and models fine‑tuned on CoVA‑SFT outperform all interleaved CoT baselines by more than 2× on average, though they still lag behind strong text‑only CoT baselines.
By Tsung-Han Wu, Heekyung Lee, Anya Ji, Haoming Chen, Trevor Darrell, Joseph E. Gonzalez, David M. Chan
Doc‑CoB introduces a Chain‑of‑Boxes framework that enhances document understanding by progressively focusing on query‑relevant layout regions while preserving global context. It selects key layout boxes and then applies visual prompting for deeper analysis, supported by two new reasoning tasks and an automatic pipeline that generates 249k training samples with intermediate visual supervision. Experiments across seven benchmarks and four popular models demonstrate significant performance gains, underscoring the method’s effectiveness and broad applicability.
By Ye Mo, Kai Ye, Xianwei Mao, Zirui Shao, Gang Huang, Bo Zhang, Hangdi Xing, Kehan Chen, Huan Zhou, Zixu Yan, Jiajun Bu, Sheng Zhou
arXiv:2607. 04625v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset.
By Junyu Xiong, Yonghui Wang, Rongjian Gu, Chenyu Liu, Bing Yin, Wengang Zhou, Houqiang Li
arXiv:2609.21675v1 Announce Type: new
Abstract: Despite the remarkable progress in Multimodal Large Language Models (MLLMs), prevailing Chain-of-Thought (CoT) paradigms remain confined to the natural...
By Wan Xu, Yuanfan Guo, Kevin Han, LaLa Chen, Wangmeng Zuo
arXiv:2606. 03273v1 Announce Type: cross Abstract: Visual DeepSearch requires multimodal large reasoning model (MLRM) agents to answer complex visual queries by repeatedly inspecting image regions, grounding intermediate reasoning in visual evidence, and connecting fine-grained clues across long reasoning chains.
By Hang He, Chuhuai Yue, Chengqi Dong, Chengcheng Wan, Ting Su, Haiying Sun, Jiajun Chai, Xiaohan Wang, Guojun Yin
Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existing multimodal RAG pipelines primarily face two critical challenges: first, standard semantic similarity retrievers frequently fetch topically overlapping yet answer-void distractor pages that mislead downstream generation; second, rigid single-pass pipelines heavily depend on initial retrieval success, where any omission of core evidence inevitably causes cascading errors.
arXiv:2608.22429v1 Announce Type: new
Abstract: Multimodal Large Language Models (MLLMs) capable of thinking with images often rely on external tools for fine-grained perception. However, this relian...
By Changjiang Jiang, Qiannian Zhao, Lei Xin, Jinxiang Xie, Preslav Nakov, Zhuohan Xie