arXiv AI

Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation

The paper introduces Evidence-Aligned Entity Verification (EAEV), a method for detecting entity-level hallucinations in retrieval-augmented generation (RAG). EAEV aligns generated entities with retrieved evidence across three dimensions and uses counterfactual stability analysis to maintain robust alignments when evidence changes. Experiments on multiple RAG benchmarks show that EAEV consistently outperforms existing hallucination detection methods and generalizes well.

arXiv AI
1d ago

Mycelium: A Meta-Study of Hallucination Detection Across Provenance, Typology, and Granularities

The paper presents a meta‑study of hallucination detection in large language models, framing it as a benchmark‑conditioned measurement problem. It organizes existing benchmarks along three axes—provenance, typology, and granularity—to analyze how construction, annotation, and evaluation choices influence detection tasks. The authors also examine span‑level evaluation, propose a new Span Coverage F1 metric, and discuss the implications for cross‑benchmark comparisons and generalization.

By Julia Belikova, Elisei Rykov
arXiv AI
Sep 10

Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation

The paper introduces Fact-Ablated Evaluation (FAE), a framework that iteratively removes cited evidence to test whether large language models (LLMs) adjust their fact‑checking predictions accordingly. Experiments reveal that many off‑the‑shelf LLMs rely more on internal knowledge than on the provided evidence. To address this, the authors propose REAL, a training method that uses counterfactual evidence supervision to encourage LLMs to base veracity judgments on evidence, achieving better evidence‑dependent performance across four datasets.

By Xingyu Deng, Mingzi Cao, Nikolaos Aletras, Xi Wang, Mark Stevenson
arXiv Computation and Language
Oct 19

Atomic Consistency Preference Optimization for Long-Form Question Answering

Atomic Consistency Preference Optimization (ACPO) is a self‑supervised preference‑tuning method that improves factual accuracy in large language models without external supervision. It uses atomic consistency signals—agreement of individual facts across multiple stochastic responses—to identify high‑ and low‑quality data pairs for model alignment. ACPO outperforms a strong supervised alignment baseline by 1.95 points on the LongFact and BioGen datasets for Phi‑3 and Llama‑3 models.

By Jingfeng Chen, Raghuveer Thirukovalluru, Junlin Wang, Kaiwei Luo, Bhuwan Dhingra