The paper investigates why vision‑language models like LLaVA‑1.5‑7B hallucinate objects in captions and proposes a targeted fix. By ranking attention heads whose image attention drops around hallucinated words, the authors identify 32 key heads and apply a head‑sliced LoRA adapter plus an inference‑time grounding controller. On COCO images, this combined method reduces hallucinated captions from 37% to 23% and hallucinated object mentions from 15.6% to 9.6%, while also lowering object recall.
arXiv:2603. 24058v2 Announce Type: replace-cross Abstract: Object hallucination in Large Vision-Language Models (LVLMs) severely compromises their reliability in real-world applications, posing a critical barrier to their deployment in high-stakes scenarios such as autonomous driving and medical image analysis.
arXiv:2608. 07302v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) often suffer from object hallucination, generating objects that are absent from the image.
arXiv:2608.21819v1 Announce Type: cross Abstract: Reliable image captioning in Vision-Language Models (VLMs) requires captions to be both precise and complete, avoiding unsupported object mentions wh...
The paper examines six inference-time hallucination mitigation methods applied to three large vision-language models across four benchmarks, including MMStar. It finds that reducing hallucination rates often comes at the cost of lower informativeness—such as decreased object recall, visual coverage, and response detail—and that gains on hallucination benchmarks do not consistently translate to improved performance on fine-grained perception and reasoning tasks. The authors argue that current evaluation protocols may overstate progress by favoring conservative generation, and propose that hallucination mitigation should be assessed as a trade-off among faithfulness, informativeness, and overall capability.
arXiv:2609.24198v1 Announce Type: cross Abstract: This paper describes the SKstars submission to SHROOM-Visions 2026, a shared task on fine-grained hallucination detection in large vision-language mo...
ReWEIGH the Evidence is a training‑free decoding technique that calibrates token‑level ordinal visual evidence to reduce hallucinations in large vision‑language models. It aggregates vocabulary ranks across visual positions, compares candidates to a token‑specific reference derived from unlabeled images, and applies a bounded penalty only when evidence falls below this reference. Experiments on four 7B backbones show up to a 21.3% reduction in hallucinated object mentions while largely preserving or improving descriptive and general performance, with minimal added latency.
The paper introduces CADMP, a lightweight framework for detecting object hallucinations in large vision‑language models. CADMP measures cross‑modal attention drift between adjacent layers and verifies predictions by masking visually relevant regions, combining these signals to identify hallucinated outputs. Experiments on multiple benchmarks show that CADMP achieves competitive detection performance, and ablation studies confirm the complementary roles of attention drift and mask‑based verification.
The paper explores soft prompting for few‑shot object detection with vision‑language models, showing that optimizing a small number of continuous prompt tokens—especially when placed at the cross‑modal boundary and initialized from an empty space token—can match LoRA performance while training far fewer parameters. Soft prompting also avoids catastrophic forgetting, transfers to newer models, and can be verbalized into readable prompts. The study extends these findings to manipulation tasks, indicating that VLMs already contain much of the necessary knowledge for specialized domains, and the main challenge is learning how to ask for it.
arXiv:2606. 07647v1 Announce Type: cross Abstract: Large vision language models (LVLMs) have made rapid advancements and are deployed across various applications, yet hallucinations remain a major challenge.
arXiv:2606. 14703v1 Announce Type: cross Abstract: How a vision-language model internally solves the task of describing an image is far from obvious.
arXiv:2609.36440v1 Announce Type: new Abstract: Large vision-language models (LVLMs) have recently achieved remarkable progress across multimodal tasks, yet object hallucination remains a persistent...
arXiv:2608.30480v1 Announce Type: cross Abstract: Object hallucination remains a persistent reliability issue in large vision-language models, where generated object mentions may sound plausible but...