arXiv Computer Vision September 16, 2026 By Truong-Binh Duong, Thanh-Ngan Tran, Ngoc-Thao Nguyen, Bac Le

Counterfactual Reasoning for Robust Visual Question Answering

Read the original on arXiv Computer Vision →

The paper introduces a new training framework for Visual Question Answering that leverages counterfactual contrastive learning to mitigate language bias and improve out‑of‑distribution generalization. It comprises a three‑stage curriculum for stable optimization, an enhanced Batch‑Contrastive loss for discriminative feature learning, and two regularizers—Answer‑Contrastive and Gradient‑Discrepancy—to refine predictions and enforce causal visual grounding. The resulting model attains 61.64% accuracy on the bias‑sensitive VQA‑CP v2 benchmark while maintaining 62.80% on the standard VQA v2 dataset, achieving a small generalization gap of 1.16%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Jun 30

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

arXiv:2606. 29984v1 Announce Type: new Abstract: Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs).

By Peng, Lee, Yin Zhang, Yanglin Zhang, Haonan Wu, Zishan Liu, Ruoxi Zang, Xin Zhu, Jiayin Zheng, Jian Yao, Zefeng Ji, Fei Ma

llms reinforcement-learning multimodal benchmarks

arXiv Computer Vision

Sep 15

Don't Just Look, Intervene: Perturbation Based Region Labeling for VQA Images

arXiv:2609.13228v1 Announce Type: new Abstract: Vision Language Models (VLMs) should rely on visual evidence that directly determines the correct answer, but supervision for grounding visual reasonin...

By Marko Jojic, Zhaonan Li, Ben Zhou

llms fine-tuning

arXiv AI

Aug 11

Evidence-RL: Towards Evidence-intensive Visual Reasoning

arXiv:2608. 08021v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) should answer from concrete image evidence rather than language priors, dataset shortcuts, or irrelevant visual context.

By Haojie Huang, Xinlei Yu, Chengming Xu, Zhangquan Chen, Cheng Yang, Qingdong He, Yu Yang, Jiangning Zhang, Xiaobin Hu

llms reinforcement-learning multimodal benchmarks

arXiv Computation and Language

Oct 27

Grounding on Request: Learning Modality Control in Vision-Language Models

arXiv:2610.33841v1 Announce Type: cross Abstract: When an image and its caption disagree, can a user reliably control which one a vision-language model (VLM) follows? Accuracy alone cannot answer thi...

By Maxime Dassen, Alexander Martin, Andrew Yates, Reno Kriz, William Walden

llms rag fine-tuning multimodal benchmarks

arXiv AI

Oct 15

The Value of Being Wrong: Self-Mined Visual In-Context Learning from Errors

arXiv:2610.16977v1 Announce Type: cross Abstract: Visual in-context learning (ICL) enables vision-language models (VLMs) to adapt at test time via demonstration examples, but its effectiveness depend...

By Shu Zhao, Tan Yu

llms rag nlp multimodal benchmarks

arXiv Computation and Language

4d ago

GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning

The paper introduces GRIT, a training framework that adds visual grounding to reinforcement learning with verifiable rewards (RLVR) for vision-language models, addressing the issue of Correct-but-Visually-Misgrounded Reasoning (CVMR). CVMR occurs when models produce correct final answers while misperceiving visual information during intermediate reasoning steps, a problem that standard RLVR rewards cannot detect. By analyzing token-level attention patterns, the authors show that CVMR tokens differ from well-grounded tokens and use this insight to guide GRIT, which improves both reasoning accuracy and grounding fidelity. Additionally, the authors present GRIT-Bench, a new benchmark for evaluating text-rich reasoning across varying text lengths and reasoning depths.

By Chanseul Cho, Noori Bae, Dongheon Lee, Youngmin Ro

llms reinforcement-learning multimodal benchmarks

One email a morning, machine-written

Related stories

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

Don't Just Look, Intervene: Perturbation Based Region Labeling for VQA Images

Evidence-RL: Towards Evidence-intensive Visual Reasoning

Grounding on Request: Learning Modality Control in Vision-Language Models

The Value of Being Wrong: Self-Mined Visual In-Context Learning from Errors

GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning