arXiv:2608. 19739v1 Announce Type: cross Abstract: Multimodal LLMs can see a document, but they often can't read it reliably.
Multimodal LLMs can see a document, but they often can't read it reliably. Small text, tables, visual cues, and topological elements still trip them up under direct visual inference, even when the pag...
arXiv:2607. 15565v1 Announce Type: cross Abstract: Where should the question go in a vision-language model (VLM) prompt: before the image or after it?
The paper investigates how visual interventions—training‑free edits applied to multimodal large language models—affect hallucinations and prediction quality. By applying a visual‑residual edit across successive decoder layers and matching its RMS strength, the authors identify an Evidence Anchoring Boundary (EAB) where the intervention’s influence drops sharply; effects before the EAB are 13.78–26.10× larger than after it. They further compare counterfactual subtraction across different question types and introduce calibrated selective routing, which only alters predictions when calibration supports an intervention, improving attribute accuracy by 1.56–1.81 percentage points while avoiding the harms of universal subtraction.
The paper introduces <flowreader>, a method that casts evidence selection for long multimodal documents as a minimum‑cost flow problem over a multimodal content graph. It uses spectral decomposition to identify latent query‑relevant aspects and allocates a fixed evidence budget proportionally to their spectral energy, ensuring aspect coverage without a language‑model planning call. On the VisDoMBench benchmark with Qwen3‑VL‑32B, <flowreader> achieves the highest macro accuracy (68.9%) and outperforms prior systems by 2.7 points, while using fewer content nodes and reader tokens.
arXiv:2606. 16494v1 Announce Type: cross Abstract: Knowledge-based visual question answering (KB-VQA) lets vision-language systems answer questions that exceed their parametric knowledge by conditioning a reader on passages retrieved from a Wikipedia-scale knowledge base.
arXiv:2610.32664v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) read images through visual tokens, and under the small per-image budgets that efficient deployment requires, t...
arXiv:2609.22628v1 Announce Type: new Abstract: Every document QA system begins with a choice that is rarely studied on its own: whether to feed the model page images, extracted text, or both. We iso...
The paper investigates the disparity between a vision‑language model’s ability to recognize text in compressed images and its ability to use that text for answering questions, a phenomenon termed the read‑use gap. Experiments show that while models can transcribe images accurately, they often fail to answer based on the transcription, and that recognition saturates at lower pixel resolutions than usage. The authors propose ALOUD, a method that allocates visual resolution to evidence detection and language tokens to reasoning, which doubles the accuracy of a frozen Qwen3‑VL‑8B on heavily compressed documents without retaining the original text.
arXiv:2607. 05438v1 Announce Type: cross Abstract: Multimodal retrieval-augmented generation (RAG) grounds a generator in evidence drawn from heterogeneous modalities -- text, tables, and images.
arXiv:2606.18974v3 Announce Type: replace Abstract: Unified multimodal models (UMMs) interleave generated ''visual thoughts'' (VTs) with text reasoning to improve spatial tasks. This incurs roughly a...
arXiv:2608. 07527v1 Announce Type: cross Abstract: Long-document understanding requires models to find and combine evidence across many pages, layouts, tables, figures, and charts.
KnowForge is a modular pipeline for open‑domain multimodal document question answering that separates retrieval, judging, and reading into distinct agents. It uses a hybrid‑retrieval knowledge‑point index, a Locator agent for searching, a Bridge Judge to enforce a second hop, a Deep Reader that handles text, images, and tables, and a synthesizer with format checking. On the M3DocVQA benchmark it achieves an F1 of 47.5, outperforming prior systems by 11 points, and shows strong gains on MultiHop‑RAG and MMLongBench‑Doc as well.