arXiv Computation and Language

Evaluating Explanation-Driven Vision-Language Reasoning via Generation Order Interventions

The paper investigates how the order of generating explanations—whether a rationale is produced before or after the answer—affects vision‑language reasoning. By conducting controlled experiments on knowledge‑intensive QA, visual entailment, and compositional grounding tasks, the authors show that larger models are required for reliable rationale‑first generation, while answer‑first generation is less susceptible to format errors. The study concludes that explanation ordering, model scale, pre‑training knowledge, fine‑tuning, and task structure jointly influence prediction accuracy and reasoning faithfulness.

arXiv AI
Sep 17

EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing

EDCT-Bench is a benchmark that evaluates the faithfulness of Vision‑Language Models (VLMs) by using Explanation‑Driven Counterfactual Testing (EDCT). EDCT extracts visual concepts from a model’s natural language explanation, applies minimal verified edits to those concepts, and checks whether the model’s answer and explanation remain consistent with the edited image. The benchmark covers three domains—knowledge‑intensive VQA, safety‑critical driving, and 3D spatial reasoning—and reveals significant faithfulness gaps in current VLMs, while also showing that EDCT‑generated counterfactuals can improve training.

By Sihao Ding, Santosh Vasa, Aditi Ramadwar, Thomas Monninger
arXiv AI
Oct 9

More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models

The paper introduces DriveMind, a large-scale driving Visual Question Answering corpus that aligns chain-of-thought reasoning with trajectory planning. Experiments with Vision‑Language Model agents show a causal disconnect: removing navigation priors significantly harms planning, while removing the chain-of-thought has little effect. The authors propose the Reasoning‑Planning Decoupling Hypothesis and present a training‑free probe to assess an agent’s reliance on priors.

By Xurui Song, Shuo Huai, JingJing Jiang, Jiayi Kong, Jun Luo
arXiv AI
Jun 10

V-REX: Benchmarking Exploratory Visual Reasoning via Chain-of-Questions

arXiv:2512. 11995v2 Announce Type: replace-cross Abstract: While many vision-language models (VLMs) are developed to answer well-defined, straightforward questions with highly specified targets, as in most benchmarks, they often struggle in practice with complex open-ended tasks, which usually require multiple rounds of exploration and reasoning in the visual space.

By Chenrui Fan, Yijun Liang, Shweta Bhardwaj, Kwesi Cobbina, Ming Li, Tianyi Zhou
arXiv Computer Vision
Oct 28

ReGuLaR: Relation-Grounded Latent Reasoning for Large Vision-Language Models

ReGuLaR introduces a relation‑grounded latent reasoning framework for large vision‑language models, explicitly anchoring latent states to key objects and their inter‑object relations. It employs a training‑time ReGFormer to focus reasoning on question‑relevant visual evidence, while inference proceeds without the ReGFormer. The authors also create the RGROUNDING‑351K dataset and demonstrate that ReGuLaR outperforms existing methods across diverse benchmarks, achieving state‑of‑the‑art results.

By Zihu Wang, Karthik Somayaji N. S, Hejia Geng, Peng Li
arXiv Computer Vision
Oct 16

AdaLoop: Thinking with the Vision Encoder

AdaLoop introduces a question‑conditioned visual recurrence framework for vision‑language models, allowing the vision encoder to adapt its computation to each question while keeping pretrained weights frozen. The method reuses encoder transformations across multiple iterations, improving performance over standard fine‑tuning and single‑pass baselines. Experiments on the VisRO dataset demonstrate gains across three VLM families on both in‑distribution and out‑of‑distribution examples.

By Xinran Nicole Han, Yiyang Lu, Katelyn Gan, Todd Zickler
arXiv Computation and Language
Oct 16

MagiC: Evaluating Multimodal Cognition Toward Grounded Visual Reasoning

MagiC is a benchmark for grounded multimodal cognition that evaluates vision‑language models on answer accuracy, step‑by‑step reasoning quality, grounding fidelity, and self‑correction. It contains about 5,500 weakly supervised QA examples generated from strong model outputs and 900 human‑curated examples with fine‑grained annotations, including rationales and bounding box groundings. The benchmark tests 15 models ranging from 7B to 70B parameters, introduces new metrics such as MagiScore and StepSense, and includes diagnostic settings to probe robustness against adversarial visual cues and introspective error correction.

By Chengfei Wu, Ronald Seoh, Bingxuan Li, Liqiang Zhang, Fengrong Han, Dan Goldwasser