arXiv Computer Vision

DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing

arXiv Computer Vision
Nov 5

DreamRealm: Exploring and Materializing Hallucination Worlds for Trustworthy Vision-Language Models

arXiv:2611.06433v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) remain prone to visual hallucinations when correct responses depend on subtle visual details. One line of post...

By Benlei Cui, Chunyang Chai, Yiliang Zhang, Xiaoqian Xia, Chengwen Yao, Song Li, Longtao Huang, Yuwen Zhai, Haiwen Hong
arXiv AI
Oct 9

Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations

The paper introduces AIMS (Adaptive Information Multi-source Steering), a lightweight, training‑free framework that dynamically balances visual, prefilled textual, and generated contexts during decoding to reduce hallucinations in Large Vision‑Language Models (LVLMs). By constructing compact prototypes for each context domain and estimating their affinities with the current query, AIMS assigns head‑wise steering weights that guide adaptive context integration without extra training or forward passes. Experiments across multiple LVLMs and decoding strategies show that AIMS effectively mitigates object hallucination while preserving general multimodal performance.

By Shuran Ma, JiaLe Li, Yuxin Dong, Shan Zheng, Qingyun Jiang, Xiang Chen, Qi Zhu, Deyi Ji, Yifan Yang, Jianfeng Pan, Yu Tian, Xue Yang
arXiv AI
Sep 15

Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations

The article surveys hallucination issues in Large Vision‑Language Models (LVLMs), a type of multimodal foundation model that blends visual data with large language models. It categorizes hallucination causes into model architecture and data quality, presents a taxonomy of mitigation strategies, and critically evaluates existing evaluation benchmarks from both discriminative and generative viewpoints. The survey also outlines open challenges and future research directions to improve LVLM reliability and trustworthiness.

By Yinghao Guo, Wei Lan, Wenyi Chen, Qingfeng Chen, Shichao Zhang, Shirui Pan, Huiyu Zhou, Yi Pan
arXiv Computer Vision
1d ago

InfiHall-Bench: A Benchmark for Hallucination Evaluation in Autoregressive Text-to-Image Generation

InfiHall-Bench is a new benchmark for evaluating hallucinations in visual autoregressive (VAR) text-to-image models, comprising 411 prompts that systematically combine five core visual skills—attribute binding, spatial relation, action relation, part consistency, and scene consistency. The benchmark reveals that VAR models exhibit persistent hallucinations across different autoregressive scales, prompting the authors to propose Confidence-Masked Hallucination Mitigation (CMHM), which focuses training on uncertain predictions and outperforms standard LoRA fine-tuning. InfiHall-Bench also demonstrates greater diagnostic power than existing diffusion-focused benchmarks by distinguishing models that appear comparable under current compositional tests.

By Yuxia Qiao, Xinhao Zhong, Junhao Li, Yi Sun, Jiaxin Hong, Shuoyang Sun, Bin Chen
arXiv Computer Vision
Aug 31

Dynamic Alignment Compensation for Hallucination Mitigation in Large Vision-Language Models

The paper introduces Dynamic Alignment Compensation (DAC), a training‑free inference‑time technique designed to reduce hallucinations in Large Vision‑Language Models (LVLMs). DAC monitors cross‑modal representation drift across decoder layers and generation steps, applying lightweight residual compensation through Layer‑wise Semantic Compensation and Sequential Semantic Correction. Experiments on nine multimodal benchmarks across various LVLM backbones demonstrate that DAC consistently lowers hallucination rates while preserving overall performance.

By Kairong Yu, Zixin Zhu, Le Yu, Hongwei Wang
arXiv Computer Vision
Sep 3

Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods

The paper examines six inference-time hallucination mitigation methods applied to three large vision-language models across four benchmarks, including MMStar. It finds that reducing hallucination rates often comes at the cost of lower informativeness—such as decreased object recall, visual coverage, and response detail—and that gains on hallucination benchmarks do not consistently translate to improved performance on fine-grained perception and reasoning tasks. The authors argue that current evaluation protocols may overstate progress by favoring conservative generation, and propose that hallucination mitigation should be assessed as a trade-off among faithfulness, informativeness, and overall capability.

By Mehrdad Fazli, Sina Mansouri, Mohit Marvania, Ziwei Zhu
arXiv AI
Jun 16

Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification

arXiv:2603. 24058v2 Announce Type: replace-cross Abstract: Object hallucination in Large Vision-Language Models (LVLMs) severely compromises their reliability in real-world applications, posing a critical barrier to their deployment in high-stakes scenarios such as autonomous driving and medical image analysis.

By Han Sun, Qin Li, Peixin Wang, Min Zhang
arXiv Computer Vision
Oct 23

MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoning

MotionHalluc is a benchmark designed to evaluate motion hallucinations in cross‑video comparison tasks, consisting of 1,540 fine‑grained questions over 553 video pairs. It assesses hallucinations across three dimensions—directional, attributional, and temporal—and reveals that current large multimodal models are highly susceptible to these errors. The authors also introduce a training‑free Perceive‑Parse‑Verify baseline that injects explicit kinematic measurements, achieving an average 10.6% performance improvement across models.

By Weile Guo, Shenghong He, Danying Mo, Chengdong Xu, Xuexun Liu, Chao Yu