DeepMind Blog
Dec 9, 2025
FACTS Benchmark Suite: Systematically evaluating the factuality of large language models
Systematically evaluating the factuality of large language models with the FACTS Benchmark Suite.
TriQua: Reconciling Granularity and Context in Factuality Evaluation
arXiv:2608. 05228v1 Announce Type: new Abstract: The "decompose-then-verify" paradigm for LLM factuality evaluation faces a fundamental trade-off: atomic facts, i.
Related stories
arXiv AI
Jun 19
CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility -- Semantic Metrics and Convergence Analysis
arXiv:2606. 19819v1 Announce Type: cross Abstract: Decomposing compound sentences into atomic, verifiable claims is a prerequisite for reliable automated fact-checking.
Hugging Face Trending Papers
Jun 21
Not All Claims Are Equally Risky: FACTOR for Adaptive Verification in Factual Long-Form Generation
Large Language Models (LLMs) generate fluent long-form text, however, often add unsupported factual claims. Existing verification techniques improve factuality by grounding generation in external evidence.
arXiv AI
Jun 26
ConflictScore: Identifying and Measuring How Language Models Handle Conflicting Evidence
arXiv:2606. 26437v1 Announce Type: cross Abstract: Existing metrics for factuality and faithfulness evaluate whether an answer is supported or contradicted by its grounding documents, but they fail to capture when both supporting and contradicting evidence coexist.
arXiv AI
Jun 16
Provenance-Enhanced Statements in Knowledge Graphs
arXiv:2606. 15246v1 Announce Type: cross Abstract: Provenance-enhanced statements of the form "according to $X$, $\varphi$" are pervasive in contemporary knowledge graphs, especially in domains where graph content primarily represents claims, interpretations, and hypotheses (\emph{capta}) rather than observer-independent facts (\emph{data}).
arXiv AI
Jun 6
PSEBench: A Controllable and Verifiable Benchmark for Evaluating LLMs in Patient Safety Event Triage
arXiv:2606. 05463v1 Announce Type: new Abstract: Patient safety event triage, determining whether a clinical event is reportable under jurisdiction-specific policy, is a high-stakes task typically performed manually by patient safety experts.
OpenAI Blog
Oct 30, 2024
Introducing SimpleQA
A factuality benchmark called SimpleQA that measures the ability for language models to answer short, fact-seeking questions.
arXiv AI
Jun 26
ReportLogic: Evaluating Logical Quality in Deep Research Reports
arXiv:2602. 18446v2 Announce Type: replace-cross Abstract: Users increasingly rely on Large Language Models (LLMs) for Deep Research, using them to synthesize diverse sources into structured reports that support understanding and action.
arXiv AI
Jun 3
When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation
arXiv:2602. 11908v3 Announce Type: replace Abstract: LLMs are widely used, yet they remain prone to factual errors that erode user trust and limit adoption in high-risk settings.
arXiv AI
Jul 31
Making Implicit Premises Explicit in Logical Understanding of Enthymemes
arXiv:2603. 06114v2 Announce Type: replace-cross Abstract: Real-world arguments in text and dialogues are normally enthymemes (i.
Hugging Face Trending Papers
Jul 30
From Single- to Cross-Document: Benchmarking Multi-Granularity Event Analysis of Large Language Models
Event analysis is an essential and fundamental direction of information extraction, involving various event-centric tasks at different granularity of documents. While large language models (LLMs) have preliminarily achieved promising performance in part of these tasks individually, their capability in event analysis still lacks comprehensive understanding due to restricted document granularity, task designs, and data source of existing benchmarks.
arXiv AI
Jun 24
Faithful by Construction: Claim-Anchored Attribution for Multi-Document Summarization
arXiv:2606. 23989v1 Announce Type: cross Abstract: End-to-end large language models (LLMs) produce fluent multi-document summaries but remain prone to hallucination, and the attributions they offer are typically coarse (whole documents or passages) and generated post hoc, leaving each summary statement hard to verify.