The paper introduces DescaPE, a decoding framework that uses internal model signals to reduce hallucinations in large language models. By identifying a factual‑salient layer span and training a lightweight probe to approximate its signal, DescaPE penalizes high‑risk continuations and rewards factually grounded ones during inference. Experiments on five factuality benchmarks across three LLMs show that DescaPE improves factuality with only a 1.10× latency overhead.
arXiv:2606. 00819v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved strong performance across diverse natural language tasks, yet their outputs often suffer from hallucinations -- content that is misaligned with factual information.
The paper introduces Variance‑Calibrated Modulation (VCM), a training‑free pre‑decoding technique that reshapes language model probability distributions before truncation. VCM uses two dynamic mechanisms: a Contextual Searchlight via PMI to suppress stopwords and highlight context‑relevant tokens, and an Adaptive Self‑Debiasing that applies scale‑invariant penalization based on real‑time logit standard deviation. Experiments on open‑ended generation, factual QA, and mathematical reasoning show that VCM consistently reduces the likelihood trap, improving diversity, coherence, and reasoning accuracy with minimal computational cost.
The paper introduces Prediction of Prediction (PoP), a method that fuses intermediate hidden representations across transformer layers during a single forward pass to detect hallucinations in large language models. PoP leverages internal hidden‑state transition dynamics to signal factual errors without extra decoding steps, achieving a 75.5% AUROC on the TruthfulQA benchmark with less than 1.2% added latency.
arXiv:2609.00796v1 Announce Type: new Abstract: As one of the most critical challenges in large language models, contextual faithfulness directly determines their reliability in knowledge-intensive a...
arXiv:2607. 01792v1 Announce Type: cross Abstract: While decoder-only LLMs excel at a vast array of natural language tasks, it suffers from an asymmetric information flow induced by causal attention: later tokens are richer in contextual grounding than earlier ones.
The paper introduces a multi‑signal pipeline for detecting hallucinations in large language model outputs, combining fine‑tuned DeBERTa‑v3 classification, Monte Carlo Dropout uncertainty, and temperature‑scaled calibration. On the HaluEval benchmark it achieves strong performance (F1 = 0.915, AUROC = 0.977) and further improves accuracy to 93.2% with MC Dropout. The authors also demonstrate that applying Direct Preference Optimization to a Qwen2.5‑0.5B generator reduces hallucination rates from 85.5% to 37.7%, and show that domain‑specific fine‑tuning (PubMedBERT on SciFact) yields better results than general‑domain training.
While decoder-only LLMs excel at a vast array of natural language tasks, it suffers from an asymmetric information flow induced by causal attention: later tokens are richer in contextual grounding than earlier ones. A simple and effective remedy is prompt repetition -- just appending a second copy of prompt before generation can redistribute grounding across positions and improve reasoning performance.
arXiv:2607. 04223v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why.
The paper introduces a multi‑signal pipeline for detecting hallucinations in large language models, combining fine‑tuned DeBERTa‑v3 classification, Monte Carlo Dropout uncertainty, and temperature‑scaled calibration. On the HaluEval benchmark it achieves high performance (F1 = 0.915, AUROC = 0.977) across QA, summarization, and dialogue, and shows that 25 % of training data yields 77 % of full‑data performance. The authors also demonstrate that applying Direct Preference Optimization to a Qwen2.5‑0.5B generator cuts hallucination rates from 85.5 % to 37.7 %, and that domain‑specific fine‑tuning (PubMedBERT on SciFact) outperforms general‑domain models for biomedical text.
arXiv:2607. 20467v1 Announce Type: new Abstract: While parallel decoding is central to the efficiency of Diffusion Large Language Models (dLLMs), current strategies are often hindered by overly conservative confidence thresholds.
arXiv:2606. 27679v1 Announce Type: cross Abstract: Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in Large Language Models (LLMs) by learning uncertainty from internal model signals.
arXiv:2606. 00301v1 Announce Type: new Abstract: Hallucinations in large language models (LLMs) arise from heterogeneous failure mechanisms, making reliable detection difficult for any single global uncertainty score.