arXiv:2603. 05308v3 Announce Type: replace-cross Abstract: Assessing whether an article supports an assertion is essential for hallucination detection and claim verification.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2607. 09349v1 Announce Type: cross Abstract: Retrieval-augmented generation evaluation checks whether model claims are factually grounded in retrieved documents.
The paper introduces REASONS, a benchmark comprising 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution by large language models. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to assess the trade-off between reliability and responsiveness. Experiments on proprietary and open-source LLMs under various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but may increase abstention, while retrieval-augmented variants often maintain near-zero abstention. Human evaluation reveals a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.
MedFabric is a new benchmark for detecting word‑level medical fabrications, comprising 646 fabricated statements each paired with a ground‑truth passage that shares the same LLM authorship and nearly identical wording. The study shows that current detectors perform poorly—expert clinicians achieve only 53.3% macro‑F1 and no detector family surpasses 60% without gold evidence—highlighting that detection hinges on evidence correctness rather than subtlety of fabrication. The authors demonstrate that a retrieval‑confidence gate can substantially improve performance, raising macro‑F1 from 61% to 74%.
The paper introduces REASONS, a benchmark of 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution under different evidence conditions. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to balance reliability and responsiveness. Experiments with proprietary and open-source LLMs across various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but increases abstention, while adversarial metadata can push hallucination rates above 85%. Human evaluation confirms a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.
arXiv:2606.21359v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses signific...
arXiv:2609.15964v1 Announce Type: new Abstract: Large language models (LLMs) have been widely adopted for clinical question answering (QA). Current systems can attach citations to their answers, but...