arXiv Computer Vision

Dynamic Alignment Compensation for Hallucination Mitigation in Large Vision-Language Models

The paper introduces Dynamic Alignment Compensation (DAC), a training‑free inference‑time technique designed to reduce hallucinations in Large Vision‑Language Models (LVLMs). DAC monitors cross‑modal representation drift across decoder layers and generation steps, applying lightweight residual compensation through Layer‑wise Semantic Compensation and Sequential Semantic Correction. Experiments on nine multimodal benchmarks across various LVLM backbones demonstrate that DAC consistently lowers hallucination rates while preserving overall performance.

arXiv AI
Jun 26

Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models

arXiv:2505. 12343v2 Announce Type: replace-cross Abstract: Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucinations, where generated content is inconsistent with the input image.

By Kai Tang, Jinhao You, Yichen Guo, Yiding Sun, Dongxu Zhang, Wenya Wang, Hanze Li, Tao Luo, Renyuan Li, Xiande Huang
arXiv AI
2d ago

Revealing Multi-View Hallucination in Large Vision-Language Models

The paper introduces the concept of multi-view hallucination (MVH), where large vision-language models produce incorrect answers when processing images from multiple viewpoints. It presents MVH-Bench, a benchmark of 4.8k question-answer pairs that target cross-instance and cross-view hallucinations, and shows that MVH is common across recent models. The authors propose Reference Shift Contrastive Decoding (RSCD), a training-free decoding method that mitigates visual interference, achieving significant performance gains on MVH-Bench with LLaVA-OneVision and Qwen2.5-VL.

By Wooje Park, Insu Lee, Soohyun Kim, Jaeyun Jang, Minyoung Noh, Kyuhong Shim, Byonghyo Shim
arXiv AI
Jun 24

When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models

arXiv:2605. 08245v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing content not present in the input.

By Harshvardhan Saini, Samyak Jha, Yiming Tang, Dianbo Liu
arXiv AI
Aug 20

ReWEIGH the Evidence: Calibrating Token-Level Ordinal Visual Evidence to Mitigate Hallucinations in Large Vision-Language Models

ReWEIGH the Evidence is a training‑free decoding technique that calibrates token‑level ordinal visual evidence to reduce hallucinations in large vision‑language models. It aggregates vocabulary ranks across visual positions, compares candidates to a token‑specific reference derived from unlabeled images, and applies a bounded penalty only when evidence falls below this reference. Experiments on four 7B backbones show up to a 21.3% reduction in hallucinated object mentions while largely preserving or improving descriptive and general performance, with minimal added latency.

By Jihae Jeong, Junha Choi, Hwanjo Yu
arXiv AI
Jun 11

MultiToP: Learning to Patch Visual Tokens to Mitigate Hallucinations in Video Large Multimodal Models

arXiv:2606. 11792v1 Announce Type: cross Abstract: Video Large Multimodal Models have achieved remarkable progress in video understanding, yet they remain prone to hallucinations, where generated responses are not faithfully supported by the input video.

By Yuansheng Gao, Wenbin Xing, Jiahao Yuan, Kaiwen Zhou, Han Bao, Zonghui Wang, Wenzhi Chen
arXiv AI
Jun 30

FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models

arXiv:2606. 29431v1 Announce Type: new Abstract: Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucination, generating content inconsistent with the input image.

By Yichen Guo, Kai Tang, Fenglai Lin, Yiding Sun, Dongshuo Zhang, Wenya Wang, Lin William Cong, Shanghang Zhang
arXiv Machine Learning
Aug 12

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

arXiv:2608. 10835v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input.

By Dvir Samuel, Guy Bar-Shalom, Fabrizio Frasca, Ethan Fetaya, Yftah Ziser, Gal Chechik, Haggai Maron