← Back to all news
arXiv AI By Amir Jalilifard, Anderson Rocha, Eric Wong, Marcos Medeiros Raimundo

Detecting Hallucination in LLMs: Tracing the Topological Signatures of Impaired Context Sharing

Read the original on arXiv AI →

The paper investigates how the topology of attention graphs can differentiate hallucinated from non-hallucinated responses in large language models. By analyzing Forman-Ricci curvature, the authors identify structural bottlenecks and develop a method that captures both semi-local and global information-flow characteristics of attention heads associated with hallucinations. Extensive evaluation across multiple LLMs and benchmarks shows that this single-pass approach consistently outperforms existing attention-based and multi-response baselines, while also revealing that impaired context sharing—such as over-reliance on self-attention and information over-squashing—correlates strongly with hallucination occurrences.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 17

Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models

The paper proposes using the temporal volatility of internal attention mechanisms—measured by an unsupervised attention dispersion metric—as a diagnostic signal for hallucinations in large language models. It demonstrates that spikes in attention entropy within intermediate layers correlate with reasoning breakdowns, and shows statistically significant AUC improvements of up to +0.076 over output-based baselines on GSM8K and MATH-500 benchmarks using the Qwen2.5 model family.

By Shardul P. More, Tanuja S. Pawar
llmsbenchmarkssafety
More like this →
Hugging Face Trending Papers
Jun 11

Layer-Resolved Optimal Transport for Hallucination Detection in NMT and Abstractive Summarization

Optimal transport (OT) has been shown to detect hallucinations in neural machine translation (NMT) by measuring the geometric distance between cross-attention distributions and a reference distribution, without any supervision. We extend this analysis to all six decoder layers of the Fairseq DE-EN model ($N=3{,}414$), showing that Wass-to-Unif and Wass-to-Data are complementary detectors specialised across hallucination types, that detection is concentrated in layers L1--L4 with L5 anti-predictive for subtler types, and that hallucinated translations lack the exploratory attention phase present in correct translations from the first decoding step.

nlpsafety
More like this →
arXiv AI
2d ago

External Observers May See More Clearly: Cross-Model Span-Level Hallucination Detection in Large Language Models via Hidden State Probing

The paper proposes a hidden‑state probing method for detecting hallucinations at the span level in large language model outputs, moving beyond token‑wise binary classification. By examining layer‑wise activation patterns, the approach identifies the exact onset and continuation tokens of hallucinations, achieving higher precision‑recall AUC than random baselines despite class imbalance. Additionally, the authors introduce a cross‑model detection framework where one model observes another’s internal representations, showing that an external observer can match or surpass the generator’s own self‑detection of hallucination onsets, even when the observer is smaller.

By Kingshuk Gupta, Davide Buscaldi
llmssafety
More like this →