arXiv AI
Sep 10

Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation

The paper introduces Evidence-Aligned Entity Verification (EAEV), a method for detecting entity-level hallucinations in retrieval-augmented generation (RAG). EAEV aligns generated entities with retrieved evidence across three dimensions and uses counterfactual stability analysis to maintain robust alignments when evidence changes. Experiments on multiple RAG benchmarks show that EAEV consistently outperforms existing hallucination detection methods and generalizes well.

By Runsong Jia, Zhen Fang, Mengjia Wu, Jie Lu, Yi Zhang
arXiv Computation and Language
Oct 21

SHAM-Bench: A Span-level Hallucination Benchmark Across Multiple Tasks

The paper introduces SHAM-Bench, a unified benchmark for detecting span-level hallucinations across six natural language generation tasks, compiled from 6,461 human-annotated examples under a common schema. It evaluates existing detection methods, finding that fine‑tuned detectors perform well in‑domain but falter under distribution shift, while prompt‑based methods with reasoning show greater robustness. The study also highlights challenges such as subtle unsupported content, ambiguous span boundaries, and noisy labels, and releases the dataset, evaluation code, and a public leaderboard for future research.

By Sicong Huang, Ian Lane