arXiv AI By Daniel Quigley, Eric Maynard

A framework for auditing grounding claims

Read the original on arXiv AI →

The paper introduces a framework for auditing grounding claims in AI systems, evaluating how tokens like "cat" are linked to real-world concepts. It reports measurements, evidence, and verdicts based on explicit acceptance criteria, assessing accuracy, robustness, and composition while documenting how mechanisms were acquired and retained. A toy gridworld example shows an agent correctly interpreting individual symbols but failing a withheld combination, highlighting a deviation from the declared composition rule and demonstrating the audit’s ability to pinpoint such issues.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Aug 25

From Subjective Judgments to Auditable Standards:Protocol-Guided AI Auditing of Website Redundancy

The paper introduces CORA (Counterfactual, Observable Redundancy Audit), a protocol for auditing website redundancy by measuring repetition load, normal-use tax, and failure-domain recovery reserve. Each audit run records screenshots, stable element identities, and task traces, while a versioned vision‑language model generates annotations that are validated and released only if they meet calibrated criteria. Experiments on a transparent mechanistic testbed show that CORA’s factorized representation separates reserve from normal-use tax and predicts perturbed success more accurately than scalar-load baselines, but it withholds automated scores when instruments fail to meet release requirements, indicating that CORA is an auditable candidate procedure for the studied benchmark rather than a universal standard.

By Ge Kong, Yongtong Cao