arXiv:2607. 18240v1 Announce Type: new Abstract: Large language models (LLMs) can achieve strong fact-checking accuracy, yet forced binary decisions conceal a critical reliability problem: systems may issue confident verdicts even when supporting evidence is weak, sparse, or internally inconsistent.
arXiv:2608.22483v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is mis...
The paper introduces a risk‑controlled framework for using large language models (LLMs) as judges in tasks without reference answers. By calibrating uncertainty thresholds on a held‑out set, the method ensures that the false discovery rate of accepted verdicts stays below a user‑specified level α with high probability, using finite‑sample Clopper–Pearson intervals. When the parametric judge lacks confidence, the instance is routed to a retrieval‑augmented mode with a second calibrated threshold, preserving the error guarantee while achieving higher coverage than single‑mode baselines.
The paper evaluates the robustness of automated fact‑checking systems by cross‑benchmarking nine models—including random baselines, fine‑tuned transformers, zero‑shot LLMs, and top AVeriTeC 2025 systems—across four datasets from scientific, open‑web, and climate domains. It finds that fine‑tuned models outperform zero‑shot LLMs on ClimateCheck, that system rankings vary strongly with domain and metric, and that replacing retrieved evidence with gold annotations boosts veracity accuracy by 14–22 points, underscoring retrieval as the main bottleneck. The authors provide code, pre‑processed datasets, and results to enable reproducible research.
Automated fact-checking (AFC) systems retrieve evidence and predict claim veracity, yet evaluations omit simple baselines, systems are developed for a single benchmark and cannot be trusted to general...
AtomCite is an agentic framework that verifies and corrects page‑level citations in multi‑page documents by parsing answers into claims, checking each claim against the cited page image, and applying a deterministic repair policy. The authors introduce DocCite, the first benchmark for this task, built on MP‑DocVQA and DUDE, containing 928 injected instances and 1,909 verified natural errors. Across Gemini, Claude, and GPT models, AtomCite achieves about 93% verification accuracy and improves citation precision from 34% to 87‑90%, while also enhancing hallucination detection in open‑source models.
arXiv:2606. 27383v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as research assistants, yet it remains unclear whether they can calibrate research takeaways to the strength and scope of the supporting evidence.
arXiv:2509. 00761v4 Announce Type: replace Abstract: Large language models are increasingly deployed for legal question answering, where evaluations typically focus on multiple-choice accuracy.
The paper introduces Fact-Ablated Evaluation (FAE), a framework that iteratively removes cited evidence to test whether large language models (LLMs) adjust their fact‑checking predictions accordingly. Experiments reveal that many off‑the‑shelf LLMs rely more on internal knowledge than on the provided evidence. To address this, the authors propose REAL, a training method that uses counterfactual evidence supervision to encourage LLMs to base veracity judgments on evidence, achieving better evidence‑dependent performance across four datasets.
AEScorer is an agentic evidence‑grounded framework designed for graded factuality verification, addressing the limitation of binary judgments in current methods. It operates in two stages: first, it gathers and refines external evidence through agentic search; second, it predicts a scalar factuality score to capture nuanced differences in correctness. The authors also introduce GradedVeriBench, a benchmark covering general and multi‑hop question answering, and demonstrate that AEScorer outperforms existing approaches on this new benchmark.
R2VC is a modular fact‑checking system that separates retrieval, reasoning, verification, and confidence calibration. It uses hybrid sparse‑plus‑dense Wikipedia retrieval, a fine‑tuned generator for structured verdicts, an NLI cross‑encoder for selecting evidence‑based candidates, and a lightweight calibrator for confidence and abstention. On the FEVER benchmark, R2VC improves accuracy by 13.74% over a baseline and shows that verifier‑based candidate selection and calibration are key contributors to performance.
arXiv:2607. 17883v1 Announce Type: cross Abstract: Enterprises will not deploy AI agents they cannot trust, and the most-cited reason for distrust is hallucination: confident, fluent output that is simply not true.
The paper introduces BioCheck Agent, an LLM-based system that generates structured biomedical fact‑checking reports using agentic search and a reinforcement‑learning framework called EG‑GRPO. Unlike prior methods that output only supported or refuted labels, BioCheck Agent synthesizes conclusions with retrieved evidence from PubMed, employing advanced Boolean search operators. Experiments show that, compared to the base Qwen3.5‑4B model, BioCheck Agent improves label prediction accuracy on SciFact by 9.95 %, raises evidence quality by 3.7 %, and reduces hallucinations by 19.63 %.