The paper introduces SHAM-Bench, a unified benchmark for detecting span-level hallucinations across six natural language generation tasks, compiled from 6,461 human-annotated examples under a common schema. It evaluates existing detection methods, finding that fine‑tuned detectors perform well in‑domain but falter under distribution shift, while prompt‑based methods with reasoning show greater robustness. The study also highlights challenges such as subtle unsupported content, ambiguous span boundaries, and noisy labels, and releases the dataset, evaluation code, and a public leaderboard for future research.
By Sicong Huang, Ian Lane
arXiv:2602.11166v2 Announce Type: replace-cross
Abstract: Parameter-efficient fine-tuning (PEFT) methods are widely used to adapt large language models (LLMs) to downstream tasks and are often assume...
By Xu Hu, Yifan Zhang, Songtao Wei, Chen Zhao, Qiannan Li, Bingzhe Li, Feng Chen
arXiv:2606. 00301v1 Announce Type: new Abstract: Hallucinations in large language models (LLMs) arise from heterogeneous failure mechanisms, making reliable detection difficult for any single global uncertainty score.
By Wentao Ye, Liyao Li, Zhiqing Xiao, Muzhi Zhu, Jiaqi Hu, Zhanming Shen, Xiaomeng Hu, Sean Du, Haobo Wang
The paper investigates whether the internal hidden states of large language models can improve the detection of hallucinations in model responses. By integrating hidden states with an output‑side baseline and evaluating on both open‑ended generation and multiple‑choice tasks, the authors find that adding hidden states yields modest AUROC gains of about 0.004 and 0.039. These results suggest that while hidden states contain additional predictive information, the overall improvement over strong standalone detection methods remains limited.
By Ningzhi Li, Xiaojun Ma, Xiaofei Wang
arXiv:2610.27247v1 Announce Type: cross
Abstract: Hallucination stress tests can yield similar average scores across language models even when different prompts elicit hallucinations in each model. W...
By Jiho Lee, Bingquan Zhang, Lin Ting Liang, Soroush Vosoughi, Weicheng Ma
arXiv:2606.21359v2 Announce Type: replace
Abstract: Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses signific...
By Raia Abu Ahmad, Nikolas Rauscher, Ekaterina Borisova, Fabio Barth, Georg Rehm, Sebastian M\"oller