The paper investigates whether vision‑language models (VLMs) can communicate visually in ways similar to humans. It reports that simple visual symbols, such as a red heart, can unexpectedly steer VLMs toward recursive solutions in unrelated tasks, revealing a generation‑interpretation gap where models produce intuitive shapes that do not effectively influence other models. In more complex communication games, VLMs can creatively use visuals to convey abstract concepts, though some messages remain unintuitive to humans, highlighting both the potential and the challenges of visual communication between humans and VLMs.
arXiv:2602. 07025v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) exhibit puzzling failures in multi-object visual tasks, such as hallucinating non-existent elements or failing to identify the most similar objects among distractions.
arXiv:2609.18345v1 Announce Type: new Abstract: Large vision-language models (LVLMs) are commonly used with only a single text prompt as the input, or plus an image. In this paper, we demonstrate tha...
The paper investigates how vision‑language models resolve conflicts between visual evidence and memorized knowledge. Using activation patching and ablation across residual streams, attention heads, and MLP sublayers, the authors find that visual grounding is the default behavior, while prior knowledge relies on a small, causally essential set of attention heads concentrated in the network’s second half. Ablating these heads flips knowledge‑grounded predictions to visually grounded ones in most cases, revealing an asymmetric causal structure and a sparse circuit of routing and writing heads that underlies perception‑knowledge conflict.
The paper investigates how vision‑language models (VLMs) resolve conflicts between image features using a Stroop‑style benchmark. By prompting contrastive and generative VLMs on images where text and color disagree, the authors find a consistent bias toward textual cues, with models preferring to respond based on text rather than color. Analysis of the embedding space reveals that textual signals are represented more strongly and separably than color signals, explaining why steering models toward text is easier than toward color.
arXiv:2608. 14286v1 Announce Type: cross Abstract: Vision language models (VLMs) are increasingly used in industrial decision-making systems, such as recruitment support and recommendation.
arXiv:2602. 02465v2 Announce Type: replace Abstract: Frontier models are transitioning from multimodal large language models (MLLMs) that merely ingest visual information to unified multimodal models (UMMs) capable of native interleaved generation.
The paper examines how humans and multimodal large language models (MLLMs) generate associative descriptions for abstract visual stimuli that become harder to name. Using two datasets—SilCC, which increases difficulty by combining abstract shapes, and PixCC, which adds visual noise—the study measures originality and effectiveness of the generated descriptions. Findings show that humans produce more diverse, semantically distant, and novel descriptions than MLLMs, and that as difficulty rises, human effectiveness drops in SilCC while they shift toward low-level visual descriptions in PixCC.
arXiv:2604. 14888v3 Announce Type: replace-cross Abstract: Recent advances in vision language models (VLMs) offer reasoning capabilities, yet how these unfold and integrate visual and textual information remains unclear.
The paper evaluates Vision Language Models (VLMs) on Visual Question Answering tasks where questions violate Grice's maxims. By generating question modifiers that add non-essential, ambiguous, or false information, the authors show that VLMs such as ChatGPT, Claude, Gemini, and Llava exhibit reduced performance. They also compare human pragmatic reasoning to VLM reasoning, noting differences in how each handles human‑induced versus AI‑generated violations, and find that humans spend less time resolving VLM‑induced violations while VLMs are less accurate in those cases.
The paper examines how multimodal language models, specifically Qwen3‑VL‑8B‑Instruct, encode different observer perspectives in their internal representations. By varying persona content and prompt framing across 100 images and 53 conditions, the study finds that early layers encode image identity while later layers develop persona‑specific structures. Prompt framing more strongly alters internal representations than persona identity, yet persona content has a larger effect on behavioural judgments, revealing a dissociation between representational geometry and functional outcomes.
arXiv:2609.16646v1 Announce Type: new Abstract: When strong multimodal models are widely available, progress requires new scientific methodologies beyond benchmark scores---using models as instrument...
arXiv:2608.22916v1 Announce Type: new Abstract: Vision-language models are known to encode spatial information in their hidden states, yet often fail to use it when answering. However, it remains unc...