arXiv:2608.30678v1 Announce Type: new Abstract: Text-rich image understanding requires multimodal large language models (MLLMs) to organize OCR (Optical Character Recognition)-grounded evidence acros...
Vision-language models (VLMs) have achieved strong performance on OCR-based benchmarks and increasingly focused on text-rich understanding, but their robustness under controlled visual degradation remains insufficiently understood. This gap is critical for OCR reasoning, where visual corruption can induce OCR errors and structural distortions, thereby introducing uncertainty into the reasoning task.
arXiv:2608. 13267v1 Announce Type: cross Abstract: Existing vision-language model (VLM) benchmarks emphasize perception and reasoning accuracy (how well VLMs describe and reason about what they see in an image), with limited attention to behavioral reliability under uncertainty (how they behave when visual evidence is missing or misleading).
The paper introduces MOTIVE, a multi-view self-verification framework for vision-language models that evaluates candidate answers from complementary perspectives and learns a reliability score aligned with correctness. By using this score to decide whether to accept an answer or trigger history-guided rethinking, MOTIVE improves reliability and efficiency compared to existing self-verification and self-correction methods. Extensive experiments on diverse multimodal benchmarks and VLM backbones show consistent performance gains and reduced unnecessary reasoning turns.
OccludeBench is a manually curated benchmark that tests vision‑language models (VLMs) on their ability to recover fully occluded objects from remaining visual evidence. The benchmark distinguishes direct cases—requiring recovery from a specific physical cue such as a shadow or reflection—from inferential cases, which rely on broader scene context. Across five VLMs, accuracy averages 42.7 % on direct cases versus 62.7 % on inferential cases, a 20‑point gap, while humans achieve 63.4 % and 68.5 % respectively with no significant gap. The study introduces OccludeTrace, a four‑stage diagnostic that isolates the failure point, revealing that models struggle most when recovering the relevant object from the full image projection. Interventions such as inpaint‑then‑confront reduce the gap to a statistically non‑significant difference for models with image‑generation capabilities.
PRISM-BENCH is a new benchmark comprising 1,000 expert‑curated vision‑language questions across five under‑represented reasoning categories, each paired with an idealized reasoning trace validated by a second annotator. It introduces diagnostics beyond final‑answer accuracy: a perception probe that attributes about 85% of frontier‑model failures to misperception, and trace alignment that predicts answer correctness with a macro AUROC of 0.66. The benchmark also presents CORE, an efficiency‑aware metric that reorders 24.2% of model pairs relative to accuracy while remaining uncorrelated with response length, revealing that recent gains are driven by recognition rather than spatial grounding.
arXiv:2508.13680v5 Announce Type: replace-cross Abstract: We introduce VMMU, a Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark designed to evaluate how vision-language models (V...
arXiv:2609.13158v1 Announce Type: new Abstract: Large Vision--Language Models (LVLMs) are increasingly expected to perform visual question answering (VQA) over planar media. However, existing planar...
The paper introduces CAIT, a benchmark of 400 synthetic scenes featuring counter‑intuitive actions that challenge multimodal large language models (MLLMs). Human participants and proprietary models like Claude and Gemini perform well, but standard open‑source instruction‑tuned MLLMs fail, largely due to a strong language prior that overrides contradictory visual evidence. The study shows that Chain‑of‑Thought reasoning can help but introduces new issues, while targeted fine‑tuning and structured prompting can reduce reliance on language priors and improve visual grounding.
arXiv:2605. 16409v3 Announce Type: replace-cross Abstract: Optical character recognition (OCR) and multilingual scene-text understanding remain challenging for multimodal large language models (MLLMs), particularly in real-world images containing small or degraded text, cluttered layouts, occlusion, handwriting, and complex typography.
arXiv:2609.37656v1 Announce Type: new Abstract: Large vision-language models (LVLMs) exhibit strong reasoning capabilities, yet the visual and textual evidence supporting the generated responses rema...
arXiv:2603. 11689v3 Announce Type: replace Abstract: Frontier Multimodal Large Language Models (MLLMs) exhibit remarkable capabilities in Visual-Language Comprehension (VLC) tasks.