arXiv AI
Causal Probing for Internal Visual Representations in Multimodal Large Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 2
Multimodal Function Vectors for Visual Relations
arXiv:2510. 02528v2 Announce Type: replace Abstract: Large Multimodal Models (LMMs) demonstrate impressive in-context learning abilities from few multimodal demonstrations, yet the internal mechanisms supporting such task learning remain opaque.
arXiv AI
Jun 8
The Geometry of Representational Failures in Vision Language Models
arXiv:2602. 07025v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) exhibit puzzling failures in multi-object visual tasks, such as hallucinating non-existent elements or failing to identify the most similar objects among distractions.
arXiv AI
Jun 3
Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models
arXiv:2605. 18160v2 Announce Type: replace-cross Abstract: In recent years, multimodal large language models (MLLMs) have achieved remarkable progress, primarily attributed to effective paradigms for integrating visual and textual information.
arXiv Computer Vision
Aug 27
Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
The paper investigates why multimodal large language models (MLLMs) struggle with vision‑centric tasks when visual evidence conflicts with pretrained language knowledge. Using image reconstruction and a new WhatIfVis benchmark, the authors show that MLLMs preserve coarse‑grained visual attributes but fail to consistently use them, and that supervised fine‑tuning and activation patching can improve controllability of visual context sensitivity. The study demonstrates that the main bottleneck lies in the models’ inability to reliably regulate their reliance on visual evidence rather than in visual perception itself.
arXiv Computation and Language
1d ago
Slow to See, Slow to Suppress: Understanding the Effects of Modality in Context-Memory Conflicts
arXiv:2609.00293v1 Announce Type: new Abstract: We investigate how vision-language models (VLMs) handle context-memory conflicts; that is, situations in which the model is given information in contex...
arXiv AI
Jun 11
From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning
arXiv:2606. 11745v1 Announce Type: cross Abstract: Visual causal reasoning is essential for understanding and intervening in the physical world, requiring identification of causal variables from visual inputs and reasoning over intervention effects.