arXiv AI

Mycelium: A Meta-Study of Hallucination Detection Across Provenance, Typology, and Granularities

The paper presents a meta‑study of hallucination detection in large language models, framing it as a benchmark‑conditioned measurement problem. It organizes existing benchmarks along three axes—provenance, typology, and granularity—to analyze how construction, annotation, and evaluation choices influence detection tasks. The authors also examine span‑level evaluation, propose a new Span Coverage F1 metric, and discuss the implications for cross‑benchmark comparisons and generalization.

arXiv Computation and Language

6d ago

SHAM-Bench: A Span-level Hallucination Benchmark Across Multiple Tasks

The paper introduces SHAM-Bench, a unified benchmark for detecting span-level hallucinations across six natural language generation tasks, compiled from 6,461 human-annotated examples under a common schema. It evaluates existing detection methods, finding that fine‑tuned detectors perform well in‑domain but falter under distribution shift, while prompt‑based methods with reasoning show greater robustness. The study also highlights challenges such as subtle unsupported content, ambiguous span boundaries, and noisy labels, and releases the dataset, evaluation code, and a public leaderboard for future research.

By Sicong Huang, Ian Lane

llms fine-tuning benchmarks safety

arXiv AI

Sep 1

Small Updates, Big Doubts: Does Parameter-Efficient Fine-tuning Enhance Hallucination Detection ?

arXiv:2602.11166v2 Announce Type: replace-cross Abstract: Parameter-efficient fine-tuning (PEFT) methods are widely used to adapt large language models (LLMs) to downstream tasks and are often assume...

By Xu Hu, Yifan Zhang, Songtao Wei, Chen Zhao, Qiannan Li, Bingzhe Li, Feng Chen

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jun 2

FLaG: Fine-Grained Latent Grouping for Hallucination Detection

arXiv:2606. 00301v1 Announce Type: new Abstract: Hallucinations in large language models (LLMs) arise from heterogeneous failure mechanisms, making reliable detection difficult for any single global uncertainty score.

By Wentao Ye, Liyao Li, Zhiqing Xiao, Muzhi Zhu, Jiaqi Hu, Zhanming Shen, Xiaomeng Hu, Sean Du, Haobo Wang

llms benchmarks safety

arXiv Computation and Language

6d ago

Detecting Errors in Large Language Model Responses: The Incremental Predictive Value of Hidden States

The paper investigates whether the internal hidden states of large language models can improve the detection of hallucinations in model responses. By integrating hidden states with an output‑side baseline and evaluating on both open‑ended generation and multiple‑choice tasks, the authors find that adding hidden states yields modest AUROC gains of about 0.004 and 0.039. These results suggest that while hidden states contain additional predictive information, the overall improvement over strong standalone detection methods remains limited.

By Ningzhi Li, Xiaojun Ma, Xiaofei Wang

llms safety

arXiv AI

6d ago

Target-Aware Curation of Hallucination Stress Tests for Large Language Models

arXiv:2610.27247v1 Announce Type: cross Abstract: Hallucination stress tests can yield similar average scores across language models even when different prompts elicit hallucinations in each model. W...

By Jiho Lee, Bingquan Zhang, Lin Ting Liang, Soroush Vosoughi, Weicheng Ma

llms safety

arXiv Computation and Language

Aug 31

Does Finetuning with Scientific Data Increase Hallucinations? A Multi-domain Factuality Evaluation of LLMs

arXiv:2606.21359v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses signific...

By Raia Abu Ahmad, Nikolas Rauscher, Ekaterina Borisova, Fabio Barth, Georg Rehm, Sebastian M\"oller

llms fine-tuning benchmarks safety

arXiv AI

Sep 10

Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation

The paper introduces Evidence-Aligned Entity Verification (EAEV), a method for detecting entity-level hallucinations in retrieval-augmented generation (RAG). EAEV aligns generated entities with retrieved evidence across three dimensions and uses counterfactual stability analysis to maintain robust alignments when evidence changes. Experiments on multiple RAG benchmarks show that EAEV consistently outperforms existing hallucination detection methods and generalizes well.

By Runsong Jia, Zhen Fang, Mengjia Wu, Jie Lu, Yi Zhang

llms rag benchmarks safety

arXiv AI

Jun 8

OpenHalDet: A Unified Benchmark for Hallucination Detection across Diverse Generation Scenarios

arXiv:2606. 06959v1 Announce Type: cross Abstract: Hallucination detection is essential for the reliable deployment of large language models (LLMs).

By Xinyi Li, Zhen Fang, Yongxin Deng, Jinyuan Luo, Hongnan Ma, Changdae Oh, Zijing Shi, Shanshan Ye, Hanchen Wang, Shu-Lin Chen, Yadan Luo, Mengyue Yang, Sean Du, Sharon Li, Ling Chen

llms benchmarks safety

Hugging Face Trending Papers

Aug 26

Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models

In 2026, we held the fourth iteration of the SHROOM Shared Task series: SHROOM-Visions (\textbf{S}hared-task on \textbf{H}allucinations and \textbf{R}elated \textbf{O}bservable \textbf{O}vergeneration...

llms nlp multimodal safety

arXiv AI

Aug 18

HalluTracer: Hallucination Detection via Depth-Averaging Truth Signals

arXiv:2608. 16353v1 Announce Type: cross Abstract: Even well-aligned large language models confidently generate factually incorrect text, making hallucination a persistent reliability risk in high-stakes deployments.

By Zhihao Guo, Zonghan Wu, Huan Huo, DaYong Ye, Junwei Zhang, Weiran Yao, Zhiwei Liu, Qingsong Wen, Yilei Shao

llms efficiency benchmarks safety

arXiv Computation and Language

Sep 22

LongNovel: A Multi-Scale Benchmark for Hallucination Detection in Long-Context Novel Summarization

arXiv:2608.18082v2 Announce Type: replace Abstract: Although context windows have expanded significantly in recent years, hallucinations in long-context summarization remain a challenge. Long novels...

By Ruizhi Zhang, Jinwei Chen, Xiangju Lu, He Yan, Mo Yu, Junmin Zhu, Wei Zhang

nlp benchmarks safety

arXiv AI

Jun 16

A Unified Definition of Hallucination: It's The World Model, Stupid!

arXiv:2512. 21577v3 Announce Type: replace-cross Abstract: Despite numerous attempts at mitigation since the inception of language models, hallucinations remain a persistent problem even in today's frontier LLMs.

By Emmy Liu, Varun Gangal, Chelsea Zou, Michael Yu, Xiaoqi Huang, Alex Chang, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng

llms rag benchmarks safety

Related stories