The paper introduces a causal and temporal evaluation framework for vision‑language models (VLMs) that tracks how visual input, question text, and generated prefixes influence autoregressive decoding. It defines three step‑indexed causal‑drive metrics—Visual Causal Drive (VCD), Question Causal Drive (QCD), and Prefix Causal Drive (PCD)—using a Structural Causal Model and interventions. Experiments on Qwen3‑VL‑8B‑Instruct and other datasets show a shift from early question and visual guidance to increased reliance on generated prefixes, and demonstrate that QCD and PCD reduce recovery error and improve bias detection.
Vision-language models (VLMs) are increasingly evaluated on complex image and video understanding tasks, yet conventional metrics primarily assess final-answer quality and reveal little about how diff...
The study examines how Vision‑Language Models (VLMs) integrate visual evidence into language‑based decisions by applying layer‑wise causal interventions on video‑text attention pathways in a video‑based generative multiple‑choice setting. Findings reveal that visual information is primarily incorporated while processing candidate answer options, with nouns serving as key semantic anchors and verbs becoming important during temporal reasoning. The research also uncovers a distinct pattern in temporal reasoning, indicating that VLMs struggle to reconstruct sequential information across video frames, possibly due to linguistic biases in temporal expressions.
arXiv:2609.22588v1 Announce Type: new Abstract: Vision-language models (VLMs) perform strongly on visual question answering benchmarks, yet often make decisions that contradict visual evidence they h...
The paper investigates how visual interventions—training‑free edits applied to multimodal large language models—affect hallucinations and prediction quality. By applying a visual‑residual edit across successive decoder layers and matching its RMS strength, the authors identify an Evidence Anchoring Boundary (EAB) where the intervention’s influence drops sharply; effects before the EAB are 13.78–26.10× larger than after it. They further compare counterfactual subtraction across different question types and introduce calibrated selective routing, which only alters predictions when calibration supports an intervention, improving attribute accuracy by 1.56–1.81 percentage points while avoiding the harms of universal subtraction.
arXiv:2609.13228v1 Announce Type: new Abstract: Vision Language Models (VLMs) should rely on visual evidence that directly determines the correct answer, but supervision for grounding visual reasonin...
The paper introduces REVEAL, a diagnostic benchmark that stresses Video‑Language Models (VidLMs) on five controlled probes—camera‑motion sensitivity, cross‑frame integration, video sycophancy, language‑only shortcuts, and temporal expectation bias—to assess how well these models encode and use visual evidence. Experiments on 12 VidLMs reveal systematic failures: some visual signals are never reliably encoded, while others are overridden by model priors, leading to performance below chance on several probes that humans solve with high accuracy. Mechanistic probes further pinpoint where and why visual evidence is lost, demonstrating that under assertive prompts a model’s output becomes nearly invariant to real versus random video input, rendering visual evidence causally inert.
arXiv:2610.19823v1 Announce Type: new Abstract: Text-to-video (T2V) generation models can produce visually realistic videos that nevertheless violate the physical consequences implied by the prompt a...
arXiv:2604. 14888v3 Announce Type: replace-cross Abstract: Recent advances in vision language models (VLMs) offer reasoning capabilities, yet how these unfold and integrate visual and textual information remains unclear.
The paper introduces Causal Visual Recurrent Reasoning (CVRR), a method that forces multimodal models to rely on latent visual states by using recurrent computation as the sole image‑conditioned path to prediction. CVRR initializes recurrence from the question hidden state after a pretrained vision‑language model has processed the image, repeatedly updates this state while re‑reading the same visual evidence, and removes all other visual traces before decoding. Experiments on several benchmarks show that CVRR maintains strong performance while other latent reasoners lose visual competence, and causal interventions confirm that predictions depend on the recurrent visual trajectory.
arXiv:2607. 13305v1 Announce Type: cross Abstract: Benchmark accuracy in video large language models (LLMs) is often treated as evidence of visual understanding.
Vision‑language models (VLMs) can lose accuracy when images are resized, even with minimal changes. The study shows that such small visual configuration changes—like tiling or token arrangement—cause more correctness flips across multiple checkpoints and benchmarks. Interestingly, in many cases the models still read the correct answer but fail to use it, and attention interventions reveal that configuration shifts weaken the use of readable information. By guiding models with field cues and their own transcriptions, the authors correct 97.2% of these errors.
arXiv:2608.22916v1 Announce Type: new Abstract: Vision-language models are known to encode spatial information in their hidden states, yet often fail to use it when answering. However, it remains unc...