arXiv Computer Vision
Nov 5

DreamRealm: Exploring and Materializing Hallucination Worlds for Trustworthy Vision-Language Models

arXiv:2611.06433v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) remain prone to visual hallucinations when correct responses depend on subtle visual details. One line of post...

By Benlei Cui, Chunyang Chai, Yiliang Zhang, Xiaoqian Xia, Chengwen Yao, Song Li, Longtao Huang, Yuwen Zhai, Haiwen Hong
arXiv AI
Oct 9

Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations

The paper introduces AIMS (Adaptive Information Multi-source Steering), a lightweight, training‑free framework that dynamically balances visual, prefilled textual, and generated contexts during decoding to reduce hallucinations in Large Vision‑Language Models (LVLMs). By constructing compact prototypes for each context domain and estimating their affinities with the current query, AIMS assigns head‑wise steering weights that guide adaptive context integration without extra training or forward passes. Experiments across multiple LVLMs and decoding strategies show that AIMS effectively mitigates object hallucination while preserving general multimodal performance.

By Shuran Ma, JiaLe Li, Yuxin Dong, Shan Zheng, Qingyun Jiang, Xiang Chen, Qi Zhu, Deyi Ji, Yifan Yang, Jianfeng Pan, Yu Tian, Xue Yang
arXiv AI
Sep 15

Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations

The article surveys hallucination issues in Large Vision‑Language Models (LVLMs), a type of multimodal foundation model that blends visual data with large language models. It categorizes hallucination causes into model architecture and data quality, presents a taxonomy of mitigation strategies, and critically evaluates existing evaluation benchmarks from both discriminative and generative viewpoints. The survey also outlines open challenges and future research directions to improve LVLM reliability and trustworthiness.

By Yinghao Guo, Wei Lan, Wenyi Chen, Qingfeng Chen, Shichao Zhang, Shirui Pan, Huiyu Zhou, Yi Pan
arXiv Computer Vision
1d ago

InfiHall-Bench: A Benchmark for Hallucination Evaluation in Autoregressive Text-to-Image Generation

InfiHall-Bench is a new benchmark for evaluating hallucinations in visual autoregressive (VAR) text-to-image models, comprising 411 prompts that systematically combine five core visual skills—attribute binding, spatial relation, action relation, part consistency, and scene consistency. The benchmark reveals that VAR models exhibit persistent hallucinations across different autoregressive scales, prompting the authors to propose Confidence-Masked Hallucination Mitigation (CMHM), which focuses training on uncertain predictions and outperforms standard LoRA fine-tuning. InfiHall-Bench also demonstrates greater diagnostic power than existing diffusion-focused benchmarks by distinguishing models that appear comparable under current compositional tests.

By Yuxia Qiao, Xinhao Zhong, Junhao Li, Yi Sun, Jiaxin Hong, Shuoyang Sun, Bin Chen