arXiv Computation and Language September 28, 2026 By Yonghong Zhang, Yong Xie, Isabel M. Parra, Ricardo Correia

Evidence-Grounded Auditing of Identification Assumptions in Climate-Policy Causal Evaluations

Read the original on arXiv Computation and Language →

The paper introduces ARGUS, a language‑model pipeline that audits evidence for identification assumptions in difference‑in‑differences studies of climate policy. ARGUS evaluates reported evidence against an eleven‑dimension rubric, abstaining when evidence cannot be retrieved. In tests, ARGUS detects 73% of injected flaws versus 18% for a keyword approach, abstains on about 40% of assessments in 26 economics papers, and often assigns higher risk than human labels in a five‑paper pilot.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

benchmarks

arXiv AI

Jul 31

Evidence-Ledger Adjudication for Claim-Evidence Traceability

arXiv:2607. 26512v1 Announce Type: new Abstract: AI agents can draft claims faster than authors can check whether the cited or retrieved evidence supports them.

By Gengyu Chen, Yongjie Yu, Weiling Wang

agents benchmarks

arXiv AI

Jun 29

CalBrief: A Pilot Diagnostic Benchmark for Evidence-Calibrated Scientific Briefing with Large Language Models

arXiv:2606. 27383v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as research assistants, yet it remains unclear whether they can calibrate research takeaways to the strength and scope of the supporting evidence.

By Yu Fu, Yongqi Kang, Yong Zhao

llms benchmarks

arXiv Machine Learning

Jul 7

Auditing the Audit: Five Failure Modes in Benchmark-Validity Audits

arXiv:2607. 02586v1 Announce Type: new Abstract: Governance frameworks ask AI providers and auditors for documented evaluation evidence, and perturbation-based construct-validity audits are a common form of that evidence.

By Yanhang Li, Zhichao Fan, Zexin Zhuang

benchmarks safety

arXiv AI

Sep 10

When Tools Hurt LLM Reasoning: State-Dependent Belief Revision under External Evidence

arXiv:2508.15754v2 Announce Type: replace-cross Abstract: Tool use is often assumed to monotonically improve reasoning, where external evidence is expected to help when relevant and be ignored when i...

By Yufeng Zhao, Junnan Liu, Hongwei Liu, Dongsheng Zhu, Yuan Shen, Songyang Zhang, Kai Chen

llms agents benchmarks

arXiv AI

Sep 10

What the Window Does Not Contain: Auditing Provenance in a Document-Grounded Instability Benchmark

arXiv:2609.06147v1 Announce Type: cross Abstract: Ask a language model the same question about the same document twenty times, and it sometimes returns two different answers. We built Probity, a benc...

By Seyed Mosayeb Alam

llms benchmarks

arXiv Computation and Language

Aug 21

When Text and Numbers Disagree: Evidence Arbitration in Large Language Models

arXiv:2608. 20116v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in settings where textual summaries, numerical observations, and external tool outputs may provide conflicting evidence.

By Mattia Carletti, Edward Phillips, Fredrik K. Gustafsson, Patitapaban Palo, Lei Clifton, Danielle Belgrave, Xiao Gu, David A. Clifton

llms benchmarks

One email a morning, machine-written

Related stories

Evidence-Ledger Adjudication for Claim-Evidence Traceability

CalBrief: A Pilot Diagnostic Benchmark for Evidence-Calibrated Scientific Briefing with Large Language Models

Auditing the Audit: Five Failure Modes in Benchmark-Validity Audits

When Tools Hurt LLM Reasoning: State-Dependent Belief Revision under External Evidence

What the Window Does Not Contain: Auditing Provenance in a Document-Grounded Instability Benchmark

When Text and Numbers Disagree: Evidence Arbitration in Large Language Models