arXiv AI
(V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreement
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jul 9
Unveiling the Visual Counting Bottleneck in Vision-Language Models
arXiv:2605. 30170v2 Announce Type: replace-cross Abstract: While Large Vision-Language Models (VLMs) excel at interpolation, they suffer catastrophic failures in systematic generalization, most notably in visual counting.
arXiv AI
Jun 19
ROSE: Benchmarking the Perception-to-Action Gap in Multimodal Models
arXiv:2606. 19965v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are increasingly expected to act on visual information, yet the same scene may require different actions under different task contexts.
arXiv AI
Sep 15
(How) Do MLLMs Report Bistable Images Like Humans?
The study investigates whether multimodal large language models (MLLMs) report bistable images, like the duck‑rabbit, in a manner similar to humans. Using the LLaVA family, researchers examined two dimensions: modulability (the influence of visual cues and linguistic priors) and exclusivity (whether responses commit to a single interpretation). Results show that both visual and linguistic manipulations shift reports in human‑consistent ways while maintaining predominantly exclusive responses, driven by competing image‑token representations and distinct bottom‑up and top‑down pathways.
arXiv Computation and Language
Oct 15
Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
arXiv:2507.12553v4 Announce Type: replace Abstract: Language models (LMs) are used for a diverse range of tasks, from question answering to writing fantastical stories. In order to reliably accomplis...
arXiv AI
Oct 27
Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability
The paper introduces Synergistic Faithfulness (𝔽_syn), a new metric based on the Shapley Interaction Index that isolates the joint contribution of visual and textual modalities in Vision‑Language Models (VLMs). It demonstrates that existing explainers overemphasize visual salience and fail to capture true cross‑modal synergy, and provides a scalable evaluation framework for auditing VLM reasoning. The authors evaluate nine XAI methods across three VLM architectures and datasets, revealing significant shortcomings in current approaches.
arXiv Computation and Language
Oct 23
Once a Tree, Always a Tree? Cross-lingual Transfer of Hierarchical Generalization in Language Models
The paper investigates whether language models can transfer hierarchical syntactic knowledge from a source language (L1) to a target language (L2) using a novel cross‑lingual poverty‑of‑stimulus (CPOS) experimental design. Results show that models can partially transfer L1 structural evidence to L2, producing a small but detectable hierarchical bias that varies across language pairs. The study further demonstrates that vocabulary overlap is crucial for successful transfer, indicating that cross‑lingual learning of hierarchical generalization is conditionally achievable.