arXiv Computation and Language
GANDR: Claim Auditing for Verifiable Legal Answer Generation
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Jul 24
Evaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis
arXiv:2607. 20527v1 Announce Type: new Abstract: Agentic LLM systems such as OpenScholar and PaperQA2 read the scientific literature and return cited answers, and both they and their benchmarks already check whether those citations hold, with a fixed attribution model or human graders.
arXiv Computation and Language
Oct 9
CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence
CiteVQA is a benchmark that evaluates multimodal large language models on both answering document questions and providing element‑level bounding‑box citations for the evidence used. It contains 1,897 questions across 711 PDFs in seven domains and two languages, with ground‑truth citations generated via an automated pipeline and strict quality control. The evaluation metric, Strict Attributed Accuracy (SAA), rewards a prediction only when both the answer and the cited region are correct, revealing widespread attribution hallucination in current models.
arXiv AI
Jul 17
L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit
arXiv:2509. 00761v4 Announce Type: replace Abstract: Large language models are increasingly deployed for legal question answering, where evaluations typically focus on multiple-choice accuracy.
arXiv AI
Oct 2
Actions with Receipts: Jointly Binding Claims, Evidence, and Execution for Replayable Tool-Agent Auditing
The paper introduces a claim‑anchored execution contract that binds a tool‑using agent’s emitted claim to its exact source span, the ordered execution prefix that produced it, and the source version and access state observed. Each receipt contains deterministic anchors, source identifiers, offsets, hashes, quotes, and a domain‑separated execution commitment, allowing a verifier to reconstruct these bindings before semantic or task labels are joined. The contract defines seven independently testable properties and demonstrates high detection rates against cross‑object attacks, with strong performance on conflict‑aware support guard evaluations.
arXiv AI
Oct 16
ClaimScope: Label-Blind Witness Verification for Mathematical Claim Auditing
ClaimScope is a 72‑task benchmark that audits mathematical claims by requiring a verdict, scope, counterexample, and repair, covering areas such as optimizer‑state transport and exact $2 imes2$ linear algebra. The study evaluates Qwen3‑8B using a witness checker that only examines the model’s submitted state, showing that verifier feedback improves counterexample acceptance and safe completions compared to generic feedback. The results demonstrate that separating claim classification from executable evidence can yield gains in witness‑structure correction without necessarily improving broader reasoning.
arXiv AI
Sep 25
Claim-Gated Source-Risk Auditing for Generative Search
The paper introduces a claim‑gated audit framework for generative search, ensuring that a query, source, and answer tuple is only considered resolved when relationship evidence, answer adoption, materiality, and disclosure are all present. It distinguishes this audit endpoint from citation support and review priority, tying decisions to versioned evidence spans and implementing a reference checker to enforce the contract. Experiments on a synthetic dataset confirm that the system correctly handles all 81 predicate combinations and rejects 192 malformed records, while ablation studies isolate endpoint logic from missing‑evidence handling.