arXiv AI

Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification

arXiv:2603. 24058v2 Announce Type: replace-cross Abstract: Object hallucination in Large Vision-Language Models (LVLMs) severely compromises their reliability in real-world applications, posing a critical barrier to their deployment in high-stakes scenarios such as autonomous driving and medical image analysis.

arXiv AI

Aug 10

Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination

arXiv:2608. 07302v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) often suffer from object hallucination, generating objects that are absent from the image.

By Zichuan Wang, Songlin Yang, Bo Peng, Zhenchen Tang, Yang Li, Beibei Dong, Jing Dong

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 9

Steer Where It Matters: Token-Level Visual-Sensitivity Steering for LVLMs Hallucination Mitigation

arXiv:2606. 07647v1 Announce Type: cross Abstract: Large vision language models (LVLMs) have made rapid advancements and are deployed across various applications, yet hallucinations remain a major challenge.

By Ruipeng Zhang, Zhihao Li, C. L. Philip Chen, Tong Zhang

llms multimodal benchmarks safety

arXiv AI

Jun 4

Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory

arXiv:2605. 24602v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood.

By Quanjiang Li, Zhiming Liu, Wei Luo, Tingjin Luo, Chenping Hou

llms multimodal benchmarks safety

arXiv AI

Jul 28

Disentangling Semantic Attention from Structural Bias in the Attention Manifold

arXiv:2607. 24017v1 Announce Type: cross Abstract: The empirical success of attention mechanism in Multimodal Large Language Models (MLLMs) often obscures its inherent, subtle flaws.

By Pengkun Jiao, Bin Zhu, Jingjing Chen, Yu-gang Jiang

llms multimodal benchmarks safety

arXiv AI

Jul 7

SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering

arXiv:2607. 04163v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have achieved remarkable progress in visual understanding tasks such as image captioning and visual question answering.

By Kai Tang, Jinhao You, Bohua Zhang, Yichen Guo, Yiding Sun, Dongxu Zhang, Chenxi Li, Xiande Huang, Shanghang Zhang

llms nlp multimodal benchmarks safety

arXiv AI

Jun 24

When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models

arXiv:2605. 08245v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing content not present in the input.

By Harshvardhan Saini, Samyak Jha, Yiming Tang, Dianbo Liu

llms rag agents fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Aug 12

UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations

arXiv:2608. 10835v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input.

By Dvir Samuel, Guy Bar-Shalom, Fabrizio Frasca, Ethan Fetaya, Yftah Ziser, Gal Chechik, Haggai Maron

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jun 29

Dismantling Pathological Shortcuts: A Causal Framework for Faithful LVLM Decoding

arXiv:2606. 27596v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) exhibit sophisticated reasoning but remain susceptible to object hallucination.

By Liu Yu, Can Chen, Ping Kuang, Zhikun Feng, Fan Zhou, Gillian Dobbie

llms multimodal benchmarks safety

arXiv AI

Jul 9

HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models

arXiv:2607. 07507v1 Announce Type: cross Abstract: Hallucinations in vision language models (VLMs) are commonly treated as semantic errors, yet they often arise from partial or ambiguous visual evidence.

By Feng He, Zhenting Wang, Qifan Wang, Qiang Guan, Dongfang Liu, Ruixiang Tang, Qiankun Li

llms multimodal safety

arXiv AI

Jul 1

ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs

arXiv:2606. 31054v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are critically hampered by hallucination, generating content inconsistent with the provided image.

By Zhiyuan Yao, Zheren Fu, Zhixiao Zheng, Jiajun Li, Yi Tu, Zhendong Mao

llms diffusion reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 30

FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models

arXiv:2606. 29431v1 Announce Type: new Abstract: Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucination, generating content inconsistent with the input image.

By Yichen Guo, Kai Tang, Fenglai Lin, Yiding Sun, Dongshuo Zhang, Wenya Wang, Lin William Cong, Shanghang Zhang

llms multimodal benchmarks safety

arXiv Machine Learning

Jul 14

SpurLens: Automatic Detection of Spurious Cues in Multimodal LLMs

arXiv:2503. 08884v3 Announce Type: replace-cross Abstract: Unimodal vision models are known to rely on spurious correlations, but it remains unclear to what extent Multimodal Large Language Models (MLLMs) exhibit similar biases despite language supervision.

By Parsa Hosseini, Sumit Nawathe, Mazda Moayeri, Sriram Balasubramanian, Soheil Feizi

llms multimodal benchmarks safety

Related stories