Post-Audit Mirage: Identifiability Limits of Offline Agent Verification

The paper introduces the concept of a "Post‑Audit Mirage," showing that offline tests can fail to detect whether an agent update will remain safe once deployed. It formalizes this limitation using matched safe and harmful lifecycles that produce identical evidence to a verifier while an isolated scorer observes opposite outcomes, and presents a theorem describing the trade‑off between error and abstention. The authors also propose a benchmark and a tool called IdentifiedRangeMonitor that attempts to recover the missing distinction in certain settings, but notes that monitoring may still be undecided for rare groups, small effects, or missing observations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• agents
• benchmarks