← Back to all news
arXiv Computer Vision By Ge Kong, Yongtong Cao

From Subjective Judgments to Auditable Standards:Protocol-Guided AI Auditing of Website Redundancy

Read the original on arXiv Computer Vision →

The paper introduces CORA (Counterfactual, Observable Redundancy Audit), a protocol for auditing website redundancy by measuring repetition load, normal-use tax, and failure-domain recovery reserve. Each audit run records screenshots, stable element identities, and task traces, while a versioned vision‑language model generates annotations that are validated and released only if they meet calibrated criteria. Experiments on a transparent mechanistic testbed show that CORA’s factorized representation separates reserve from normal-use tax and predicts perturbed success more accurately than scalar-load baselines, but it withholds automated scores when instruments fail to meet release requirements, indicating that CORA is an auditable candidate procedure for the studied benchmark rather than a universal standard.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Aug 26

Confidently Wrong, Silently So: Auditing Undetectable Failures of a Deployed On-Device Language Model

The paper audits a developer‑accessible on‑device language model, revealing that it can confidently produce incorrect answers while refusing benign prompts, a phenomenon termed task‑asymmetric miscalibration. The model’s confident outputs are surface‑indistinguishable, with classifiers based on user‑visible features failing to separate correct from wrong responses. The authors propose a model‑agnostic audit protocol, a surface‑indistinguishability test, and a black‑box consistency wrapper that improves reliability without requiring model access.

By Shashwat Pandey, Satwik Pandey, Suresh Raghu
llmssafety
More like this →
arXiv Machine Learning
Aug 26

PinSieve: Production Selective VLM Serving and a Governed Memory Flywheel for Enterprise Content-Quality Triage

PinSieve is a production system that selectively serves vision‑language models (VLMs) for enterprise content‑quality triage, operating only on the grey‑zone cases that lightweight models cannot resolve. The deployed VLM Serving Agent filters 2.05× more non‑actionable items, improves review productivity by 25.7%, cuts operating costs by 16.2%, and delivers signals the same day instead of the next. A governed memory flywheel with selective feedback, audit sampling, and a bounded proposal‑verifier loop further reduces false‑negative rates from 17.73% to 13.29% over six months, while a reasoning review agent audits teacher‑generated rationales for keep/repair/drop decisions. whyItMatters":"The system demonstrates how selective VLM serving and governed feedback loops can substantially improve efficiency, cost, and accuracy in enterprise AI content‑quality pipelines."

By Chuqing Gao, Yuanfang Song, Jonathan Zhang, Yifan Wu, Vishwakarma Singh, Qinglong Zeng, Andrey Gusev
agentsmultimodalsafety
More like this →