The paper introduces a causal and temporal evaluation framework for vision‑language models (VLMs) that tracks how visual input, question text, and generated prefixes influence autoregressive decoding. It defines three step‑indexed causal‑drive metrics—Visual Causal Drive (VCD), Question Causal Drive (QCD), and Prefix Causal Drive (PCD)—using a Structural Causal Model and interventions. Experiments on Qwen3‑VL‑8B‑Instruct and other datasets show a shift from early question and visual guidance to increased reliance on generated prefixes, and demonstrate that QCD and PCD reduce recovery error and improve bias detection.
By Shuyao Xiao, Shengling Wang, Haoyu Niu, Ke Chao, Changwei Xu, Xinran Duan, Chaoyong Jiang
The study examines how Vision‑Language Models (VLMs) integrate visual evidence into language‑based decisions by applying layer‑wise causal interventions on video‑text attention pathways in a video‑based generative multiple‑choice setting. Findings reveal that visual information is primarily incorporated while processing candidate answer options, with nouns serving as key semantic anchors and verbs becoming important during temporal reasoning. The research also uncovers a distinct pattern in temporal reasoning, indicating that VLMs struggle to reconstruct sequential information across video frames, possibly due to linguistic biases in temporal expressions.
By Davide Testa, Hugh Mee Wong, Alessandro Lenci, Bernardo Magnini, Albert Gatt
arXiv:2609.22588v1 Announce Type: new
Abstract: Vision-language models (VLMs) perform strongly on visual question answering benchmarks, yet often make decisions that contradict visual evidence they h...
By Yuyang Dai, Bofei Huang, Hongbo Zhang, Haoran Xie
The paper introduces REVEAL, a diagnostic benchmark that stresses Video‑Language Models (VidLMs) on five controlled probes—camera‑motion sensitivity, cross‑frame integration, video sycophancy, language‑only shortcuts, and temporal expectation bias—to assess how well these models encode and use visual evidence. Experiments on 12 VidLMs reveal systematic failures: some visual signals are never reliably encoded, while others are overridden by model priors, leading to performance below chance on several probes that humans solve with high accuracy. Mechanistic probes further pinpoint where and why visual evidence is lost, demonstrating that under assertive prompts a model’s output becomes nearly invariant to real versus random video input, rendering visual evidence causally inert.
By Sethuraman T V, Savya Khosla, Aditi Tiwari, Vidya Ganesh, Rakshana Jayaprakash, Aditya Jain, Vignesh Srinivasakumar, Onkar Kishor Susladkar, Srinidhi Sunkara, Aditya Shanmugham, Rakesh Vaideeswaran, Abbaas Alif Mohamed Nishar, Simon Jenni, Rohan Maheshwari, Derek Hoiem
The paper investigates how visual interventions—training‑free edits applied to multimodal large language models—affect hallucinations and prediction quality. By applying a visual‑residual edit across successive decoder layers and matching its RMS strength, the authors identify an Evidence Anchoring Boundary (EAB) where the intervention’s influence drops sharply; effects before the EAB are 13.78–26.10× larger than after it. They further compare counterfactual subtraction across different question types and introduce calibrated selective routing, which only alters predictions when calibration supports an intervention, improving attribute accuracy by 1.56–1.81 percentage points while avoiding the harms of universal subtraction.
By Qiji Zhou, Junyi Hu, Peiyu Wang, Yue Zhang
arXiv:2609.13228v1 Announce Type: new
Abstract: Vision Language Models (VLMs) should rely on visual evidence that directly determines the correct answer, but supervision for grounding visual reasonin...
By Marko Jojic, Zhaonan Li, Ben Zhou
arXiv:2607. 17523v1 Announce Type: cross Abstract: Recent advances in world models and video generation have given rise to an emerging reasoning paradigm that leverages video generative models to simulate, predict, and reason about real-world dynamics.
By Yongheng Zhang, Guang Yang, Ruihan Hou, Qiguang Chen, Ziang Liu, Xiaolong Liu, Manman Zhang, Yanchao Hao, Zheng Wei, Hao Wu, Libo Qin, Peishan Dai, Yinghui Li, Di Yin, Xing Sun
arXiv:2610.19823v1 Announce Type: new
Abstract: Text-to-video (T2V) generation models can produce visually realistic videos that nevertheless violate the physical consequences implied by the prompt a...
By Siwei Meng, Xuerou Cai, Rui Hu, Ping Liu
arXiv:2608.22916v1 Announce Type: new
Abstract: Vision-language models are known to encode spatial information in their hidden states, yet often fail to use it when answering. However, it remains unc...
By Zeyu Wang, Xinming Xu
arXiv:2606. 27596v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) exhibit sophisticated reasoning but remain susceptible to object hallucination.
By Liu Yu, Can Chen, Ping Kuang, Zhikun Feng, Fan Zhou, Gillian Dobbie
Vision‑language models (VLMs) can lose accuracy when images are resized, even with minimal changes. The study shows that such small visual configuration changes—like tiling or token arrangement—cause more correctness flips across multiple checkpoints and benchmarks. Interestingly, in many cases the models still read the correct answer but fail to use it, and attention interventions reveal that configuration shifts weaken the use of readable information. By guiding models with field cues and their own transcriptions, the authors correct 97.2% of these errors.
By Dingyang Lin, Yingfeng Luo, Chenglong Wang, Chenwei Zhu, Anxiang Ma, Jingbo Zhu, Tong Xiao
arXiv:2607. 13305v1 Announce Type: cross Abstract: Benchmark accuracy in video large language models (LLMs) is often treated as evidence of visual understanding.
By Jae Joong Lee