arXiv Computer Vision

Semantic-Spatial Agreement Verification for Mitigating Object Hallucination in Multimodal Large Language Models

arXiv AI
Sep 24

RelCheck: Dual-Evidence Spatial Grounding for VLM Hallucination Correction

RelCheck is a training‑free post‑hoc correction pipeline that addresses relational hallucinations in multimodal large language models. It augments object‑level visual grounding with two forms of relational evidence—learned scene‑graph triples from RelTR and deterministic spatial predicates derived from bounding‑box geometry—forming a three‑layer visual knowledge base. When applied to LLaVA v1 13B, RelCheck improves the overall MME hallucination score from 585.0 to 630.0, with the most significant gain on spatial position accuracy.

By Siddhi Patil, Navrati Saxena, William B. Andreopoulos
arXiv AI
Jul 21

Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain

arXiv:2603. 21693v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have shown strong potential for medical Visual Question Answering (VQA), yet they remain prone to hallucinations, defined as generating responses that contradict the input image, posing serious risks in clinical settings.

By Mohammad Asadi, Tahoura Nedaee, Jack W. O'Sullivan, Euan Ashley, Ehsan Adeli
Hugging Face Trending Papers
Jun 14

Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference

Multimodal large language models (MLLMs) have demonstrated strong capabilities in vision-language understanding and natural-language response generation. However, these systems can still produce overconfident predictions and hallucination-like outputs, particularly when the visual evidence is weak, ambiguous, or semantically inconsistent.

Hugging Face Trending Papers
Jul 29

Explainable and Resource-Efficient Spatial Reasoning in Multimodal LLMs for Decision-Critical Applications

As Multimodal Large Language Models (MLLMs) are increasingly deployed in decision-critical pipelines such as robotics, embodied AI, and safety monitoring, the opacity of their spatial judgments limits operator trust and auditability. MLLMs demonstrate strong reasoning but often struggle with fine-grained spatial understanding and object hallucination.

arXiv Computation and Language
1d ago

VEPO: Mitigating Multilingual Hallucinations in LVLMs via Visual Evidence-guided Preference Optimization

VEPO (Visual Evidence-guided Preference Optimization) is a training-time method designed to reduce object hallucinations in large vision-language models, especially for low-resource languages. It localizes key visual evidence using English queries and creates preference pairs that differ in evidence sufficiency, encouraging models to favor answers supported by complete visual evidence. Experiments on multilingual datasets (POPE and MME) with Chinese, French, and Hindi demonstrate consistent hallucination reduction and improved perception, with the greatest gains in the lowest-resource language.

By Chengpeng Fu, Xiaocheng Feng, Wenshuai Huo, Yichong Huang, Baohang Li, Duyu Tang, Ting Liu
arXiv Computer Vision
6d ago

Mixture of Cognitive Experts in Large Vision-Language Models

The paper proposes a framework that integrates multiple specialized computer‑vision experts into large vision‑language models (LVLMs) by using a hierarchical, evidence‑driven reasoning protocol inspired by Bloom’s taxonomy. It introduces a two‑stage cognitive verbalization process: first, a Literal Evidence Summary decomposes expert outputs into atomic evidence statements; second, Bloom Verbalization converts these into a staged reasoning trace. A lightweight Reasoning Trace Module then quantitatively analyzes the trace, making evidence usage and reasoning progression explicit, leading to improved perception and reasoning performance and revealing query‑dependent cognitive entry levels and evidence‑use trajectories.

By Robert Wijaya, Ngai-Man Cheung