arXiv Computation and Language

Knowing Isn't Always Saying: When Do Spatial Encodings Reach Answers in Vision-Language Models?

arXiv Computer Vision
Sep 3

Who Drives the Probability Game of VLMs? A Temporal Causal Drive Evaluation Framework

The paper introduces a causal and temporal evaluation framework for vision‑language models (VLMs) that tracks how visual input, question text, and generated prefixes influence autoregressive decoding. It defines three step‑indexed causal‑drive metrics—Visual Causal Drive (VCD), Question Causal Drive (QCD), and Prefix Causal Drive (PCD)—using a Structural Causal Model and interventions. Experiments on Qwen3‑VL‑8B‑Instruct and other datasets show a shift from early question and visual guidance to increased reliance on generated prefixes, and demonstrate that QCD and PCD reduce recovery error and improve bias detection.

By Shuyao Xiao, Shengling Wang, Haoyu Niu, Ke Chao, Changwei Xu, Xinran Duan, Chaoyong Jiang
arXiv AI
Aug 10

Probing Visual Concepts in Lightweight Vision-Language Models for Automated Driving

arXiv:2603. 06054v2 Announce Type: replace-cross Abstract: The use of Vision-Language Models (VLMs) in automated driving applications is becoming increasingly common, with the aim of leveraging their reasoning and generalisation capabilities to handle long-tail scenarios.

By Nikos Theodoridis, Reenu Mohandas, Ganesh Sistu, Anthony Scanlan, Ciar\'an Eising, Tim Brophy
arXiv AI
Jul 15

Visual Access Boundaries in Vision-Language Model Reasoning

arXiv:2607. 12815v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting is widely used as a test-time scaling strategy for Vision-Language Models (VLMs), but it remains unclear what is extended when VLMs generate longer reasoning traces.

By Hiroto Osaka, Shohei Taniguchi, Gouki Minegishi, Kai Yamashita, Masahiro Suzuki, Yutaka Matsuo
arXiv Computation and Language
Sep 7

From Vision to Language: Investigating Causal Information Flow in Multimodal Decision-Making

The study examines how Vision‑Language Models (VLMs) integrate visual evidence into language‑based decisions by applying layer‑wise causal interventions on video‑text attention pathways in a video‑based generative multiple‑choice setting. Findings reveal that visual information is primarily incorporated while processing candidate answer options, with nouns serving as key semantic anchors and verbs becoming important during temporal reasoning. The research also uncovers a distinct pattern in temporal reasoning, indicating that VLMs struggle to reconstruct sequential information across video frames, possibly due to linguistic biases in temporal expressions.

By Davide Testa, Hugh Mee Wong, Alessandro Lenci, Bernardo Magnini, Albert Gatt