arXiv Computer Vision September 16, 2026 By Truong-Binh Duong, Thanh-Ngan Tran, Ngoc-Thao Nguyen, Bac Le

Counterfactual Reasoning for Robust Visual Question Answering

Read the original on arXiv Computer Vision →

The paper introduces a new training framework for Visual Question Answering that leverages counterfactual contrastive learning to mitigate language bias and improve out‑of‑distribution generalization. It comprises a three‑stage curriculum for stable optimization, an enhanced Batch‑Contrastive loss for discriminative feature learning, and two regularizers—Answer‑Contrastive and Gradient‑Discrepancy—to refine predictions and enforce causal visual grounding. The resulting model attains 61.64% accuracy on the bias‑sensitive VQA‑CP v2 benchmark while maintaining 62.80% on the standard VQA v2 dataset, achieving a small generalization gap of 1.16%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Jun 30

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

arXiv:2606. 29984v1 Announce Type: new Abstract: Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs).

By Peng, Lee, Yin Zhang, Yanglin Zhang, Haonan Wu, Zishan Liu, Ruoxi Zang, Xin Zhu, Jiayin Zheng, Jian Yao, Zefeng Ji, Fei Ma

llms reinforcement-learning multimodal benchmarks

arXiv Computer Vision

3d ago

Don't Just Look, Intervene: Perturbation Based Region Labeling for VQA Images

arXiv:2609.13228v1 Announce Type: new Abstract: Vision Language Models (VLMs) should rely on visual evidence that directly determines the correct answer, but supervision for grounding visual reasonin...

By Marko Jojic, Zhaonan Li, Ben Zhou

llms fine-tuning

arXiv AI

Aug 11

Evidence-RL: Towards Evidence-intensive Visual Reasoning

arXiv:2608. 08021v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) should answer from concrete image evidence rather than language priors, dataset shortcuts, or irrelevant visual context.

By Haojie Huang, Xinlei Yu, Chengming Xu, Zhangquan Chen, Cheng Yang, Qingdong He, Yu Yang, Jiangning Zhang, Xiaobin Hu

llms reinforcement-learning multimodal benchmarks

arXiv AI

Aug 10

Debias in Text, Believe Your Eyes: Text-Anchored Cross-Modal Transfer for Visual Counter-Commonsense Reasoning

arXiv:2608. 06938v1 Announce Type: cross Abstract: The visual reasoning ability of multimodal large language models (MLLMs) is crucial for downstream applications, particularly counter-commonsense reasoning, which requires models to reason beyond common assumptions.

By Chen Ling, Hanqian Li, Dongnan Liu, Keyu Qian, Jungang Li, Xinglong liu, Shiyi Wang, Xin Dong, Pengcheng Zhu, Wei Zhou, Linjian Mo, Nai Ding

llms efficiency multimodal benchmarks

arXiv Machine Learning

Jun 8

Diagnosing Visual Ignorance in Vision-Language Models

arXiv:2606. 06890v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) frequently rely on language priors, producing confident answers that are weakly grounded in visual evidence.

By Runyu Zhou, Qi Zhang, Qixun Wang, Yisen Wang

llms multimodal benchmarks

arXiv AI

1d ago

EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing

EDCT-Bench is a benchmark that evaluates the faithfulness of Vision‑Language Models (VLMs) by using Explanation‑Driven Counterfactual Testing (EDCT). EDCT extracts visual concepts from a model’s natural language explanation, applies minimal verified edits to those concepts, and checks whether the model’s answer and explanation remain consistent with the edited image. The benchmark covers three domains—knowledge‑intensive VQA, safety‑critical driving, and 3D spatial reasoning—and reveals significant faithfulness gaps in current VLMs, while also showing that EDCT‑generated counterfactuals can improve training.

By Sihao Ding, Santosh Vasa, Aditi Ramadwar, Thomas Monninger

llms nlp fine-tuning multimodal benchmarks safety

One email a morning, machine-written

Related stories

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

Don't Just Look, Intervene: Perturbation Based Region Labeling for VQA Images

Evidence-RL: Towards Evidence-intensive Visual Reasoning

Debias in Text, Believe Your Eyes: Text-Anchored Cross-Modal Transfer for Visual Counter-Commonsense Reasoning

Diagnosing Visual Ignorance in Vision-Language Models

EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing