arXiv AI
Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 7
Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering
arXiv:2604. 01280v2 Announce Type: replace-cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires Multimodal Large Language Models (MLLMs) to identify and combine fine-grained visual cues with retrieved textual evidence.
arXiv AI
Jun 17
See First, Answer Later: Visual Evidence Pre-Alignment via Sufficiency-Driven RL
arXiv:2606. 17678v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images, indicating ineffective utilization of visual evidence during inference.
arXiv Machine Learning
Jul 9
Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering
arXiv:2607. 07179v1 Announce Type: cross Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents.
arXiv AI
Aug 10
Debias in Text, Believe Your Eyes: Text-Anchored Cross-Modal Transfer for Visual Counter-Commonsense Reasoning
arXiv:2608. 06938v1 Announce Type: cross Abstract: The visual reasoning ability of multimodal large language models (MLLMs) is crucial for downstream applications, particularly counter-commonsense reasoning, which requires models to reason beyond common assumptions.
arXiv AI
Aug 25
SAFE-G: Structure-aware Faithful Evidence-guided Generation for Knowledge-based Visual Question Answering
arXiv:2608.21796v1 Announce Type: cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) aims to answer queries that necessitate reasoning over external knowledge sources beyond the visua...
arXiv AI
Sep 25
Mind What Matters for Reasoning: Aligning Cross-Modal Attention via Selective Probability Mass Concentration
The paper introduces Selective Probability Mass Concentration (sPMC), a training framework that strengthens implicit visual grounding in multimodal large language models by selectively regularizing attention heads most responsive to visual evidence. sPMC treats attention over visual tokens as a spatial probability distribution and encourages mass to concentrate on semantically relevant regions using segmentation-derived priors, while leaving other heads unconstrained. Across six multimodal benchmarks, sPMC yields an average zero‑shot improvement of 3% and gains up to 11.3% for various models by regularizing only 3%–15% of their attention heads.