arXiv AI

Attention-space Contrastive Guidance for Efficient Hallucination Mitigation in LVLMs

arXiv Computation and Language

Oct 15

Mitigating Video Hallucinations via Attention-Guided Representation Perturbation

arXiv:2610.19728v1 Announce Type: cross Abstract: Video large language models (Video LLMs) often generate responses unsupported by the input video. Contrastive decoding (CD) can mitigate these halluc...

By Zhixin Cai, Ruoqi Yuan, Xinchen Yang, Zhuofan Chen, Jun Bai, Chuantao Yin, Wenge Rong

llms benchmarks safety

arXiv AI

Sep 2

Revealing Multi-View Hallucination in Large Vision-Language Models

The paper introduces the concept of multi-view hallucination (MVH), where large vision-language models produce incorrect answers when processing images from multiple viewpoints. It presents MVH-Bench, a benchmark of 4.8k question-answer pairs that target cross-instance and cross-view hallucinations, and shows that MVH is common across recent models. The authors propose Reference Shift Contrastive Decoding (RSCD), a training-free decoding method that mitigates visual interference, achieving significant performance gains on MVH-Bench with LLaVA-OneVision and Qwen2.5-VL.

By Wooje Park, Insu Lee, Soohyun Kim, Jaeyun Jang, Minyoung Noh, Kyuhong Shim, Byonghyo Shim

llms multimodal benchmarks safety

arXiv Computation and Language

Oct 9

SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoders

The paper introduces SAGE, a lightweight method that guides decoder attention in vision-language models away from prompt‑invariant attention sinks (PIS) toward query‑dependent regions of interest (ROIs). By using token‑aligned ROI masks from vision backbones like CLIP, ViT, and DINOv3, SAGE improves visual grounding and reduces hallucinations across various VLM families. Experiments show consistent gains on public vision‑language benchmarks, especially in fine‑grained visual discrimination tasks where localized evidence is essential.

By Jeonghyo Song, YoungJoon Yoo

llms multimodal benchmarks safety

arXiv AI

Jun 24

When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models

arXiv:2605. 08245v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing content not present in the input.

By Harshvardhan Saini, Samyak Jha, Yiming Tang, Dianbo Liu

llms rag agents fine-tuning multimodal benchmarks safety

arXiv AI

Jul 1

ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs

arXiv:2606. 31054v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are critically hampered by hallucination, generating content inconsistent with the provided image.

By Zhiyuan Yao, Zheren Fu, Zhixiao Zheng, Jiajun Li, Yi Tu, Zhendong Mao

llms diffusion reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 30

FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models

arXiv:2606. 29431v1 Announce Type: new Abstract: Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucination, generating content inconsistent with the input image.

By Yichen Guo, Kai Tang, Fenglai Lin, Yiding Sun, Dongshuo Zhang, Wenya Wang, Lin William Cong, Shanghang Zhang

llms multimodal benchmarks safety

Hugging Face Trending Papers

Sep 29

Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery

While Large Vision-Language Models (LVLMs) achieve remarkable success, hallucinations remain a significant barrier to their reliable deployment. Recent studies primarily attribute these issues to cros...

llms multimodal benchmarks safety

arXiv Computer Vision

Sep 30

Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery

arXiv:2609.37263v1 Announce Type: new Abstract: While Large Vision-Language Models (LVLMs) achieve remarkable success, hallucinations remain a significant barrier to their reliable deployment. Recent...

By Siqi Lu, Suo Wei, Yongbin Zheng, Jianhang Yao, Wanying Xu, Peng Wang

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 9

Steer Where It Matters: Token-Level Visual-Sensitivity Steering for LVLMs Hallucination Mitigation

arXiv:2606. 07647v1 Announce Type: cross Abstract: Large vision language models (LVLMs) have made rapid advancements and are deployed across various applications, yet hallucinations remain a major challenge.

By Ruipeng Zhang, Zhihao Li, C. L. Philip Chen, Tong Zhang

llms multimodal benchmarks safety

arXiv AI

Jul 28

Disentangling Semantic Attention from Structural Bias in the Attention Manifold

arXiv:2607. 24017v1 Announce Type: cross Abstract: The empirical success of attention mechanism in Multimodal Large Language Models (MLLMs) often obscures its inherent, subtle flaws.

By Pengkun Jiao, Bin Zhu, Jingjing Chen, Yu-gang Jiang

llms multimodal benchmarks safety

arXiv AI

Jun 4

Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory

arXiv:2605. 24602v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood.

By Quanjiang Li, Zhiming Liu, Wei Luo, Tingjin Luo, Chenping Hou

llms multimodal benchmarks safety

arXiv AI

Aug 10

Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination

arXiv:2608. 07302v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) often suffer from object hallucination, generating objects that are absent from the image.

By Zichuan Wang, Songlin Yang, Bo Peng, Zhenchen Tang, Yang Li, Beibei Dong, Jing Dong

llms multimodal benchmarks safety

Related stories