arXiv Computation and Language
Aug 27

Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models

In 2026, the SHROOM-Visions shared task was launched at the UncertaiNLP Workshop co‑located with EMNLP to address hallucinations in large vision‑language models. The task builds on the SHEEP dataset and asks participants to detect and classify fine‑grained hallucination spans in image‑conditioned text generation across four languages (Chinese, English, French, Italian) using a five‑class taxonomy. The competition attracted 27 teams and over 600 system submissions, with top systems achieving character‑level, label‑conditioned, and IoU scores of 0.58, 0.46, and 0.51 respectively, surpassing baselines by 30‑40 points.

By Ra\'ul V\'azquez, Aman Sinha, Chuyuan Li, Claudio Savelli, Eduardo Cal\`o, Emilio Raimond, Stella Frank, Hengyu Luo, Flavio Giobergia, Vincent Segonne, Lorenzo Vaiani, J\"org Tiedemann, Timothee Mickus
arXiv Computation and Language
Sep 1

SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models

SpanCalib-VLM is a hybrid system for detecting hallucinated text spans in Vision‑Language Models. It combines a multimodal sequence tagger (XLM‑RoBERTa‑Large + SigLIP) with a fine‑tuned generative VLM (Qwen3.5‑4B‑SHROOM‑SFT) and uses a Union‑Calibrated Fusion strategy to re‑score candidate spans. On the SHROOM‑Visions English evaluation split, the ensemble achieves a Pearson calibration correlation of 0.41, an overall IoU of 0.39, a clean‑response IoU of 0.91, and a detection accuracy of 70.7%.

By Amanuel Gizachew Abebe, Yasmin Moslem
arXiv Computation and Language
6d ago

Sckwoky at SHROOM-Visions: Character-Level Hallucination-Span Detection in Image-Conditioned VLMs via Dual-Prompted LoRA Decoders

The paper describes a system for detecting character‑level hallucinations in image‑conditioned vision‑language model responses, evaluated on the SHROOM‑Visions 2026 shared task across English, French, Italian, and Chinese. The detector uses a frozen Gemma‑4‑12B‑it backbone with ~10.7 M trainable LoRA adapters, repeats the candidate answer in a single prompt to provide bidirectional context, and combines hidden states to predict hallucination probabilities, BIO boundaries, and abstention scores, with language‑specific classifiers assigning hallucination types. On the closed test set the system achieves a macro‑averaged character IoU of 0.4404, more than double the baseline, and ranks 5th‑6th across languages, with ablations showing the importance of answer repetition and image‑answer pairing.

By Mikhail Kulakov (Novosibirsk State University), Ruslan Solikhov (Novosibirsk State University), Ivan Bondarenko (Novosibirsk State University)