MedFabric is a new benchmark for detecting word‑level medical fabrications, comprising 646 fabricated statements each paired with a ground‑truth passage that shares the same LLM authorship and nearly identical wording. The study shows that current detectors perform poorly—expert clinicians achieve only 53.3% macro‑F1 and no detector family surpasses 60% without gold evidence—highlighting that detection hinges on evidence correctness rather than subtlety of fabrication. The authors demonstrate that a retrieval‑confidence gate can substantially improve performance, raising macro‑F1 from 61% to 74%.
arXiv:2607. 20462v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly integrated into clinical workflows, stressing the need for reliable traceability of model-generated output with watermarking.
arXiv:2603. 05308v3 Announce Type: replace-cross Abstract: Assessing whether an article supports an assertion is essential for hallucination detection and claim verification.
arXiv:2607. 09349v1 Announce Type: cross Abstract: Retrieval-augmented generation evaluation checks whether model claims are factually grounded in retrieved documents.
MedRAGChecker is a claim-level verification framework designed for biomedical retrieval‑augmented generation (RAG). It decomposes generated answers into atomic claims and assesses each claim’s support by combining evidence‑grounded natural language inference with biomedical knowledge‑graph consistency signals. The aggregated claim decisions provide diagnostics that distinguish retrieval and generation failures, such as faithfulness, under‑evidence, contradiction, and safety‑critical errors, and the system is distilled into compact models for scalable evaluation.
The paper investigates how misleading context—specifically fabricated evidence and bare assertions—affects large language models’ medical question‑answering performance. Experiments on MedMisBench show that models are more prone to adopt answers based on assertions than fabricated evidence, and that these misleading cues are often disclosed in reasoning traces but rarely in final responses. A monitor that reads open reasoning traces can detect most corrupted decisions, whereas monitoring only responses is less effective.
arXiv:2606. 09500v1 Announce Type: new Abstract: Objective.
The paper introduces BioCheck Agent, an LLM-based system that generates structured biomedical fact‑checking reports using agentic search and a reinforcement‑learning framework called EG‑GRPO. Unlike prior methods that output only supported or refuted labels, BioCheck Agent synthesizes conclusions with retrieved evidence from PubMed, employing advanced Boolean search operators. Experiments show that, compared to the base Qwen3.5‑4B model, BioCheck Agent improves label prediction accuracy on SciFact by 9.95 %, raises evidence quality by 3.7 %, and reduces hallucinations by 19.63 %.
arXiv:2603. 09986v3 Announce Type: replace-cross Abstract: Hallucinations, the tendency for large language models to provide responses with factually incorrect and unsupported claims, is a serious problem within natural language processing for which we do not yet have an effective solution to mitigate against.
arXiv:2606. 29034v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly summarize clinical evidence, where a claim's weight depends on how strongly it is supported.
arXiv:2606.21359v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses signific...
arXiv:2608. 16643v1 Announce Type: cross Abstract: Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such models rest on benchmarks that evaluate each clinical note in isolation.
arXiv:2608. 09080v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks.