The paper introduces DISCERN, a two-tier protocol for certifying that updates to production models do not increase risk. It first uses unlabeled data to detect benign updates based on disagreement rates, then selectively labels only disagreements through an anytime-valid confidence sequence. The method achieves finite-sample validity with label-complexity bounds of order ρ²/ε², demonstrating significant label savings and strong empirical performance across 14,000+ audit streams.
LLM-generated records increasingly populate production information systems where they are treated as data assets subject to the same quality expectations as records from operational databases. A singl...
The paper introduces BiasGuard‑D, a system that uses two adversarial language‑model agents from different families to exchange typed claim documents, with a third model quorum voting to adjudicate. A combined referential‑plus‑semantic invariant validates the verdict, and the iterative deliberation layer shifts remediation from stereotypes to hallucinations, improving right‑answer recovery by 49 percentage points on HaluEval QA. All claims, votes, and instructions are stored as typed records, providing an auditable provenance trail for operators under emerging AI regulations.
arXiv:2610. 37055v1 Announce Type: cross Abstract: Audit-based certification bounds the violation rate of an LLM agent by auditing a random fraction $\varepsilon$ of actions and reweighting the audited labels; its validity is meant to rest only on the defender's audit coin.
arXiv:2605.29139v2 Announce Type: replace-cross Abstract: Question-answering services built on retrieval-augmented generation (RAG), in which a language model answers from retrieved documents, are in...
CascadeLedger is a counterfactual execution‑audit framework designed for budgeted language‑model cascades. It enforces call‑before‑read semantics, records service and cache identities, and freezes event traces before scoring, ensuring that two executions can share a final answer and resource projection only if their routing decisions used legitimately observable information. The framework introduces counterfactual prefix non‑interference (CPNI), counterfactual influence frontier (CIF), and projection‑preserving adversarial mutation (PPAM) to detect and localize violations, achieving a clean false‑positive rate and high detection rates for serialized and hidden‑read faults while quantifying overhead for online and offline audits.
arXiv:2608. 16852v1 Announce Type: new Abstract: Regulatory compliance monitoring in deployed language models is increasingly implemented as a legal and audit control, checking model outputs against written rules spanning data protection, healthcare, financial regulation, and platform policy.
arXiv:2610. 10597v1 Announce Type: cross Abstract: Public leaderboards for AI models are read continuously, and attackers can see every published standing.
The paper introduces a framework for evaluating AI systems that not only checks final labels but also tracks the reasoning behind them through three core sources—grounds, norms, and authority—forming an eight-cell counterfactual judgment cube. It defines minimal source replacement sets, called judgment receipts, to explain changes in verdicts and provides certification cost bounds for black-box evaluators. The authors present ReasonBench, a benchmark with 19,520 cases, and demonstrate that while high standard accuracy can mask robustness issues, receipt accuracy reveals significant gaps in reasoning consistency across different models.
The paper introduces ClaimReceipt, a specification and verifier that checks whether a claim in an agent evaluation can be recomputed from retained evidence (sufficiency) and whether the evidence covers the entire experiment set (coverage). Using the CR‑2 verifier on 1,392 historical records, the authors demonstrate accurate reproduction of audit verdicts, non‑redundant field groups, and zero false positives on semantic faults. In a prospective CR‑3 run, the system correctly flags missing receipts and preserves coverage when private evidence is withheld, while adding minimal overhead to inference time and transaction size.
arXiv:2610.10199v1 Announce Type: new Abstract: Emerging legislation requires large language models (LLMs) to be audited for compliance with regulatory standards, particularly fairness. Such black-bo...
JuryProbe is an empirical diagnostic tool designed to assess consensus risk in panels of reference‑free large language model judges used for factuality verification. It estimates risk by measuring false‑negative correlations and false‑consensus lift from a labeled calibration probe, and routes high‑risk majority decisions to judges with trusted references. The approach was validated on FEVER corruptions, showing that flagged decisions can be grounded without additional reference acquisition in most cases, while reducing false accepts by about 0.4% and avoiding 28% of reference acquisitions.
arXiv:2609.13714v1 Announce Type: new Abstract: An updated model can improve an aggregate metric while degrading a slice that matters to a downstream user. We study checkpoint selection subject to no...