arXiv AI
Jun 19
The Hidden Evolution of Disguised Visual Context inside the VLM
arXiv:2606. 20077v1 Announce Type: cross Abstract: Visual tokens enter Large Language Models (LLMs) as raw, foreign signals.
Listening makes Vision Clear for VLMs
arXiv:2606. 23763v1 Announce Type: cross Abstract: Recent work typically assesses vision--language consistency using attention distributions of answer-side tokens.
Related stories
arXiv AI
Jun 8
Focus-then-Context: Subject-Centric Progressive Visual Token Reduction for Vision-Language Models
arXiv:2605. 20950v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) face a bottleneck of prohibitive computational costs arising from massive visual token sequences during inference.
arXiv AI
Jun 3
When Attention Collapses: Stage-Aware Visual Token Pruning from Structure to Semantics
arXiv:2606. 03569v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated remarkable capabilities but suffer from significant computational overhead during inference.
Hugging Face Trending Papers
Jul 22
Test-Time Training for Modality Order Consistency in Vision-Language Models
We find that vision-language models are sensitive to a specific semantically irrelevant change: the order in which the image and question are presented. Across three models and three benchmarks, image first prompting consistently outperforms question-first prompting, revealing a repeatable modality order failure.
arXiv AI
Jul 28
Disentangling Semantic Attention from Structural Bias in the Attention Manifold
arXiv:2607. 24017v1 Announce Type: cross Abstract: The empirical success of attention mechanism in Multimodal Large Language Models (MLLMs) often obscures its inherent, subtle flaws.
Hugging Face Trending Papers
Jun 2
When Attention Collapses: Stage-Aware Visual Token Pruning from Structure to Semantics
Vision-Language Models (VLMs) have demonstrated remarkable capabilities but suffer from significant computational overhead during inference. While visual token pruning offers a promising solution, existing methods predominantly rely on initial attention scores.
arXiv AI
Jul 15
Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models
arXiv:2509. 22415v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved strong vision-language performance, yet their token-level visual evidence remains difficult to inspect.
arXiv AI
Aug 10
Learning to Predict Middle-Layer Attention in MLLMs for Visual Token Prunin
arXiv:2608. 06411v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) achieve strong performance across diverse vision-language tasks, but their efficiency is limited by the cost of processing numerous visual tokens.
Hugging Face Trending Papers
Jul 7
AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring
Vision-Language Models (VLMs) are promising for construction-site monitoring, and recent construction-tailored VLMs have primarily adapted pretrained VLMs through direct QA-style fine-tuning from a single global image. We argue that this direct paradigm remains limited for in-the-wild deployment in terms of operational range, reliability under reduced-resolution inputs, and inference efficiency.
arXiv AI
Jun 16
Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification
arXiv:2603. 24058v2 Announce Type: replace-cross Abstract: Object hallucination in Large Vision-Language Models (LVLMs) severely compromises their reliability in real-world applications, posing a critical barrier to their deployment in high-stakes scenarios such as autonomous driving and medical image analysis.
arXiv AI
Jul 7
Token-Based Affordance Grounding with Large Vision-Language Models
arXiv:2607. 03595v1 Announce Type: cross Abstract: Affordance grounding aims to localize image regions that support a specific action, serving as a core capability for physical intelligence and embodied perception.
arXiv AI
Jul 8
Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention
arXiv:2607. 05978v1 Announce Type: cross Abstract: Multimodal large language models can emit localized predictions, bounding boxes for objects and temporal windows for video and audio events, but they hallucinate these regions prolifically.