arXiv AI September 1, 2026 By Joonki Min, Chaeyun Kim, Hyungwook Choi, Yejin Kim, Kihyun Kim, Yohan Jo, Joonseok Lee

Fine-Grained Multi Image Object Hallucination Benchmark

The paper introduces MIOH, a fine‑grained benchmark for evaluating object hallucination in multimodal large language models (MLLMs) across multiple images. It assesses hallucination through four tasks—existence, counting, attribute, position—using three reasoning patterns (comprehensive, comparative, selective) and three adversarial pressures (visual context scale, perceptual difficulty, contextual bias). Evaluation of 29 models, including GPT‑5 and Gemini‑2.5‑Pro, shows distinct failure patterns, indicating that hallucination arises from integration‑stage limitations rather than just perceptual errors.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Aug 31

Fine-Grained Multi Image Object Hallucination Benchmark

Multimodal Large Language Models (MLLMs) are increasingly deployed in multi-image scenarios requiring complex reasoning across visual contexts. However, current MLLMs remain fundamentally limited by o...

llms multimodal benchmarks safety

arXiv Computer Vision

Sep 2

Beyond Language Priors: Diagnosing and Fixing Visual-Origin Hallucinations in Multimodal LLM

arXiv:2609.00231v1 Announce Type: new Abstract: Existing research on object hallucination in multimodal large language models (MLLMs) predominantly attributes the problem to language priors such as o...

By Peiyang Xu, Xiaopei Zhu, Jun Zhu, Xiaolin Hu

llms rag fine-tuning multimodal benchmarks safety

arXiv AI

Jul 9

HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models

arXiv:2607. 07507v1 Announce Type: cross Abstract: Hallucinations in vision language models (VLMs) are commonly treated as semantic errors, yet they often arise from partial or ambiguous visual evidence.

By Feng He, Zhenting Wang, Qifan Wang, Qiang Guan, Dongfang Liu, Ruixiang Tang, Qiankun Li

llms multimodal safety

arXiv Computation and Language

Sep 11

OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

OmniHallu is a unified framework for detecting hallucinations in multimodal large language models across both comprehension and generation tasks involving image, video, and audio modalities. It introduces OmniHallu-Bench, a 10,000-sample benchmark with claim-level human annotations for six cross-modal tasks (I2T, V2T, A2T, T2I, T2V, T2A). The system uses a multi‑agent architecture that decomposes outputs into atomic claims, verifies them with modality‑specific experts, and aggregates evidence through structured reasoning, while a preference‑optimized verifier reduces expert calls by 66% with minimal performance loss.

By Jianjiang Yang, Peihang Li, Shanqing Xu, Mengchen Qian, Lu Zhang, Meng Luo

llms agents diffusion multimodal benchmarks safety

arXiv Computer Vision

Sep 15

VIGIL: Tackling Hallucination Detection in Image Recontextualization

arXiv:2602.14633v3 Announce Type: replace Abstract: We introduce VIGIL (Visual Inconsistency & Generative In-context Lucidity), a benchmark dataset and framework that provides a fine-grained categori...

By Joanna Wojciechowicz, Maria {\L}ubniewska, Jakub Antczak, Justyna Baczy\'nska, Wojciech Gromski, Wojciech Koz{\l}owski, Maciej Zieba

multimodal benchmarks safety

arXiv Computer Vision

Oct 8

Dual-Pathway Circuits of Object Hallucination in Vision-Language Models

arXiv:2605.13156v2 Announce Type: replace Abstract: Vision-language models (VLMs) have demonstrated remarkable capabilities in bridging visual perception and natural language understanding, enabling...

By Jiaxin Liu, Ding Zhong, Yue Wang, Zhidong Yang, Zhaolu Kang, Guangyuan Dong, Qishi Zhan, Pengcheng Fang, Aofan Liu

llms multimodal safety

One email a morning, machine-written

Related stories

Fine-Grained Multi Image Object Hallucination Benchmark

Beyond Language Priors: Diagnosing and Fixing Visual-Origin Hallucinations in Multimodal LLM

HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models

OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

VIGIL: Tackling Hallucination Detection in Image Recontextualization

Dual-Pathway Circuits of Object Hallucination in Vision-Language Models