arXiv AI
Aug 7
TriQua: Reconciling Granularity and Context in Factuality Evaluation
arXiv:2608. 05228v1 Announce Type: new Abstract: The "decompose-then-verify" paradigm for LLM factuality evaluation faces a fundamental trade-off: atomic facts, i.
ElementCheck: Complexity-Aware Long-Form Text Factuality Evaluation via Sentence Elements
ElementCheck is a new framework for evaluating the factuality of long-form text that addresses limitations of the traditional decompose‑retrieve‑verify pipeline. Rather than breaking sentences into atomic sub‑claims, it extracts entity pairs linked by verifiable connections to form an element graph, using the graph’s topology to gauge sentence complexity. This allows simple sentences to be verified directly while complex ones undergo targeted element‑level refinement, and the authors introduce the FastFact‑Sent benchmark to support fine‑grained evaluation, demonstrating consistent improvements across multiple backbone models.
Related stories
Hugging Face Trending Papers
Jun 21
Not All Claims Are Equally Risky: FACTOR for Adaptive Verification in Factual Long-Form Generation
Large Language Models (LLMs) generate fluent long-form text, however, often add unsupported factual claims. Existing verification techniques improve factuality by grounding generation in external evidence.
arXiv AI
Aug 28
FaithSieve: Fine-Grained Evaluation of Math Proofs with Faithful Formal Evidence
FaithSieve is a Lean‑assisted framework that fine‑grains natural‑language mathematical proofs into local reasoning units, extracts typed proof obligations, and verifies them with formal evidence gated by semantic alignment. It introduces two expert‑verified datasets—ProofLoc‑Olympiad and ProofLoc‑University—to benchmark first‑error localization. On these benchmarks, FaithSieve outperforms direct‑judging baselines, achieving 81.43% and 84.5% exact first‑error accuracy respectively.
arXiv Computation and Language
Sep 17
ProofVerifier: A Scalable, Diversity-Driven Framework for Natural-Language Proof Verification
arXiv:2602.02377v3 Announce Type: replace Abstract: While large language models (LLMs) have achieved strong performance on mathematical problems with verifiable answers, many advanced problems are pr...
arXiv AI
Sep 7
Attributable by Construction: Claim-Anchored Provenance for Multi-Document Summarization
The paper introduces CAMS, a Claim‑Anchored Multi‑Document Summarization framework that decomposes source documents into atomic claims, resolves provenance deterministically from verbatim quotes to token spans, clusters equivalent claims across documents, and rewrites summaries so each sentence ends with claim identifiers linking back to source spans. CAMS separates provenance (an invariant for each emitted sentence) from faithfulness (an objective encouraged by selection, rewriting, and verification). Evaluations on MultiNews, DiverseSumm, and zero‑shot WCEP show that CAMS matches strong baselines in summary quality while improving faithfulness and citation precision, raising attribution accuracy from 38% to 64% and reducing human verification time per claim by 3.4×.
arXiv AI
Sep 10
Evaluating and Improving Evidence-Grounded Fact-Checking in LLMs via Multi-Round Evidence Ablation
The paper introduces Fact-Ablated Evaluation (FAE), a framework that iteratively removes cited evidence to test whether large language models (LLMs) adjust their fact‑checking predictions accordingly. Experiments reveal that many off‑the‑shelf LLMs rely more on internal knowledge than on the provided evidence. To address this, the authors propose REAL, a training method that uses counterfactual evidence supervision to encourage LLMs to base veracity judgments on evidence, achieving better evidence‑dependent performance across four datasets.
arXiv AI
Jun 19
CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility -- Semantic Metrics and Convergence Analysis
arXiv:2606. 19819v1 Announce Type: cross Abstract: Decomposing compound sentences into atomic, verifiable claims is a prerequisite for reliable automated fact-checking.
Hugging Face Trending Papers
Jul 9
Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench
Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them. However, existing long-context evaluations - from Needle-in-a-Haystack (NIAH) tests to more recent multi-hop reasoning and summarization tasks - predominantly measure average-case performance, and many are either saturated or lack robustness.
arXiv AI
Aug 11
Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation
arXiv:2510. 21891v2 Announce Type: replace-cross Abstract: To deploy large language models (LLMs) in high-stakes application domains that require substantively accurate responses to open-ended prompts, we need reliable, computationally inexpensive methods that assess the trustworthiness of long-form responses generated by LLMs.
arXiv AI
Jul 10
Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench
arXiv:2607. 08284v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them.
arXiv Machine Learning
Sep 22
EAVer: Long-Form Factuality Verification as an End-to-End Agentic Policy
arXiv:2609.22223v1 Announce Type: cross Abstract: Long-form factuality verification is commonly implemented as a static decompose-search-verify pipeline, with separately prompted modules processing c...
arXiv AI
Jun 9
Inference-Time Conformal Reasoning with Valid Factuality Control for Large Language Models
arXiv:2606. 08831v1 Announce Type: new Abstract: Large language models (LLMs) increasingly perform multi-step reasoning, where intermediate claims form implicit directed acyclic graphs whose node correctness is structurally conditioned on their ancestors.