← Back to all news
arXiv Computer Vision By Kairong Yu, Zixin Zhu, Le Yu, Hongwei Wang

Dynamic Alignment Compensation for Hallucination Mitigation in Large Vision-Language Models

Read the original on arXiv Computer Vision →

The paper introduces Dynamic Alignment Compensation (DAC), a training‑free inference‑time technique designed to reduce hallucinations in Large Vision‑Language Models (LVLMs). DAC monitors cross‑modal representation drift across decoder layers and generation steps, applying lightweight residual compensation through Layer‑wise Semantic Compensation and Sequential Semantic Correction. Experiments on nine multimodal benchmarks across various LVLM backbones demonstrate that DAC consistently lowers hallucination rates while preserving overall performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Sep 15

Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations

The article surveys hallucination issues in Large Vision‑Language Models (LVLMs), a type of multimodal foundation model that blends visual data with large language models. It categorizes hallucination causes into model architecture and data quality, presents a taxonomy of mitigation strategies, and critically evaluates existing evaluation benchmarks from both discriminative and generative viewpoints. The survey also outlines open challenges and future research directions to improve LVLM reliability and trustworthiness.

By Yinghao Guo, Wei Lan, Wenyi Chen, Qingfeng Chen, Shichao Zhang, Shirui Pan, Huiyu Zhou, Yi Pan
llmsmultimodalbenchmarkssafetydata-engineering
More like this →
arXiv AI
Jun 26

Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models

arXiv:2505. 12343v2 Announce Type: replace-cross Abstract: Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucinations, where generated content is inconsistent with the input image.

By Kai Tang, Jinhao You, Yichen Guo, Yiding Sun, Dongxu Zhang, Wenya Wang, Hanze Li, Tao Luo, Renyuan Li, Xiande Huang
llmsfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 9

From Suppression to Repair: Mitigating Object Hallucination in Large Vision-Language Models via Localized Distribution Alignment

The paper introduces ResOT, a training‑free method that mitigates object hallucination in large vision‑language models by repairing hidden representations at inference time. ResOT aligns hallucinated directions with faithful ones through localized distribution alignment and Gaussian optimal transport, creating a low‑dimensional residual subspace for minimal intervention. Experiments on three LVLMs demonstrate significant reductions in hallucination and improvements in image caption quality and multimodal performance.

By Chen Zhao, Xingping Dong, Jiachun Shi, Liang Peng, Chong Wang, Zhen Lei, Ran He, Bo Du
llmsmultimodalbenchmarkssafety
More like this →
arXiv AI
Sep 2

Revealing Multi-View Hallucination in Large Vision-Language Models

The paper introduces the concept of multi-view hallucination (MVH), where large vision-language models produce incorrect answers when processing images from multiple viewpoints. It presents MVH-Bench, a benchmark of 4.8k question-answer pairs that target cross-instance and cross-view hallucinations, and shows that MVH is common across recent models. The authors propose Reference Shift Contrastive Decoding (RSCD), a training-free decoding method that mitigates visual interference, achieving significant performance gains on MVH-Bench with LLaVA-OneVision and Qwen2.5-VL.

By Wooje Park, Insu Lee, Soohyun Kim, Jaeyun Jang, Minyoung Noh, Kyuhong Shim, Byonghyo Shim
llmsmultimodalbenchmarkssafety
More like this →