arXiv:2607. 10400v1 Announce Type: cross Abstract: Vision language models (VLMs) have achieved strong performance on visual document understanding benchmarks such as DocVQA, ChartQA, and MMLongBench-Doc.
By Abhigya Verma, Khyati Mahajan, Amit Kumar Saha, Shruthan Radhakrishna, Sagar Davasam, Vikas Yadav, Sai Rajeswar Mudumba
The paper surveys multimodal speculative decoding, examining whether diffusion-based block‑parallel generative drafting—successful in text‑only LLMs—can be applied to Vision‑Language, Video‑Language, Audio, and Vision‑Language‑Action models. It introduces a taxonomy separating drafter‑side parallelism from other design choices, and presents an empirical comparison across benchmarks such as OCR, VQA, visual reasoning, and image captioning. The study highlights current limitations, outlines open challenges, and suggests future research directions for multimodal speculative decoding.
By Yantao Li, Huanlin Gao, Fang Zhao, Chao Tan, Qiang Hui, Shuting Liu, Fuyuan Shi, Ting Lu, Shaoan Zhao, Xueqiang Guo, Xinpei Su, Jianbing Zhang, Xinyu Dai, Kai Wang, Shiguo Lian
MIDR (Multimodal Indexing for Document Retrieval) is a training‑free framework that enriches document indexes by converting rendered pages into verified textual fields with a multimodal LLM, then indexing those fields with BM25F and optionally fusing with dense retrieval. By shifting multimodal reasoning to index time, MIDR enables text‑centric serving while retaining multimodal evidence, achieving a 23.0% relative gain over BM25 on ViDoRe V3 and outperforming ColQwen2.5 on several domains with significantly smaller index memory and lower query latency.
By Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro, Yong Zhuang, Ozan Irsoy
arXiv:2606. 16092v1 Announce Type: cross Abstract: Real-world documents combine text with tables, charts, photographs, and diagrams arranged in diverse layouts, yet existing research on multimodal large language models (MLLMs) for document QA predominantly produces text-only responses, underutilizing these visual elements.
By Young Rok Jang, Hyesoo Kong, Kyunghwan An, Jae Sub Huh, Gyeonghun Kim, Stanley Jungkyu Choi
Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance often relies on costly on-server Large Multimodal Models (LMMs), while compact locally deployable models lack sufficient KIE supervision. We present SAYRE, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design.
The paper introduces Synth-JDoc, a synthetic Japanese document image dataset created by rendering text with HTML and CSS to produce multi‑column layouts that include both vertical and horizontal writing styles. Images generated by text‑to‑image models are embedded to enhance visual realism, and noise and degradation filters are applied to improve robustness. Experiments show that fine‑tuning Large Vision Language Models on Synth‑JDoc yields superior performance on reading vertically written Japanese text compared to prior synthetic datasets.
By Keito Sasagawa, Shuhei Kurita, Daisuke Kawahara