arXiv AI August 24, 2026 By Souptik Kumar Majumdar, Fabian K\"ogel, Andreas Bulling

CARD: Diagnosing Belief to Action Routing Failures in Vision Language Models

The paper introduces CARD, a diagnostic tool that steers activations along one mental-state axis while measuring predictions on another, to investigate how vision‑language models use internal representations of beliefs, knowledge, and intentions. Using CARD on open‑weight VLMs evaluated on the Relay Chain cooperative grid‑world benchmark, the authors uncover a routing failure: models do not incorporate belief representations into their next action predictions, leaving partner information unused.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision

Sep 22

What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior

arXiv:2609.24576v1 Announce Type: cross Abstract: Modern Vision-Language Navigation (VLN) models rely mostly on pre-trained large Vision-Language Models (VLMs) to predict navigation actions. While th...

By D\'ebora Oliveira Makowski, Samiran Gode, Abhijeet Nayak, Marco Hutter, Cordelia Schmid, Lukas Rosenberger Schmid, Wolfram Burgard

llms agents fine-tuning multimodal safety

arXiv AI

Sep 1

When Should Models Change Their Minds? Contextual Belief Management in Large Language Models

arXiv:2605.30219v2 Announce Type: replace Abstract: Long-horizon interactions require language models to manage accumulating information: when to update their state, when to preserve their state, and...

By Haoming Xu, Weihong Xu, Zongrui Li, Mengru Wang, Yunzhi Yao, Chiyu Wu, Jin Shang, Yu Gong, Shumin Deng

llms reinforcement-learning benchmarks

arXiv Machine Learning

Aug 21

Answer-Level Trust Selection for Physical Vision-Language Reasoning

arXiv:2608. 19807v1 Announce Type: new Abstract: Vision-language models (VLMs) can estimate physical quantities such as duration, speed, and acceleration from visual observations, but existing benchmarks primarily assess overall model performance against annotated ground truth.

By Rongyu Yu, Ke Niu, Fengxiang He

llms fine-tuning multimodal benchmarks

Hugging Face Trending Papers

Sep 15

ENCP: Episode-Normalized Conformal Prediction for Vision-and-Language Navigation

Uncertainty estimation for Vision-Language-Navigation (VLN) models is a critical task since it can help identify ambiguous and unreliable predictions, enabling agents to make safer navigation decision...

agents multimodal

arXiv AI

5d ago

The belief-action gap in language model agents

arXiv:2610.17444v1 Announce Type: cross Abstract: Language model (LM) agents can fail for different reasons, requiring different explanations and interventions. We study how state beliefs, action val...

By Ayse Sila Okcu, Weisheng Wang, Tongmeng Xie, Niall Dalton, Angelos Nalmpantis, Raghu Arghal, Mario Giulianelli

llms agents reinforcement-learning

arXiv AI

Jun 19

Tri-Info: Generalizable, Interpretable Failure Prediction for VLA Models via Information Theory

arXiv:2606. 19998v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models are increasingly deployed across diverse tasks, yet they remain black boxes whose physical interactions can cause irreversible harm, making generalizable and interpretable failure detection essential.

By Jinghan Yang, Yunchao Zhang, Wang Yuan, Haolun Wan, Jiaming Zhang, Zhengyang Hu, Yanchao Yang

robotics multimodal benchmarks

One email a morning, machine-written

Related stories

What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior

When Should Models Change Their Minds? Contextual Belief Management in Large Language Models

Answer-Level Trust Selection for Physical Vision-Language Reasoning

ENCP: Episode-Normalized Conformal Prediction for Vision-and-Language Navigation

The belief-action gap in language model agents

Tri-Info: Generalizable, Interpretable Failure Prediction for VLA Models via Information Theory