When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors
Read the original on arXiv Computation and Language →The paper introduces EICS, a metric for assessing whether Vision‑Language Models (VLMs) consistently encode concepts under controlled one‑concept interventions, and CASP‑VL, a benchmark of causally controlled spatial perturbations derived from Visual Genome scene graphs. Experiments across multiple VLM architectures reveal that spatial relations are encoded far less faithfully than lexical information, even when models perform well on compositional benchmarks. The study shows that relational information is retained at the predicate token level but disappears when sentences are aggregated into a global representation, highlighting a systematic failure invisible to output‑based evaluation.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.