arXiv Machine Learning
Jul 31
What Must a Fairness Audit Report When Demographic Data Is Incomplete?
arXiv:2506. 23033v5 Announce Type: replace Abstract: Fairness audits are a key component of responsible machine-learning deployment.
How Reliable are Fairness Audits with Unreliable Data?
arXiv:2506. 23033v2 Announce Type: replace Abstract: Fairness audits are a key component of responsible machine-learning deployment.
Related stories
arXiv AI
Jun 3
The Reliability Gap in Benchmark Auditing: Distribution Shift and Scale as Failure Modes of Contamination Detection
arXiv:2606. 03305v1 Announce Type: new Abstract: Benchmark contamination, where evaluation examples appear in a model's training data, threatens the validity of LLM assessment.
arXiv Machine Learning
Jun 16
Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget
arXiv:2606. 16110v1 Announce Type: new Abstract: Machine unlearning has been extensively studied in response to growing privacy concerns and regulatory requirements.
arXiv AI
1d ago
What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models
arXiv:2608. 16852v1 Announce Type: new Abstract: Regulatory compliance monitoring in deployed language models is increasingly implemented as a legal and audit control, checking model outputs against written rules spanning data protection, healthcare, financial regulation, and platform policy.
arXiv AI
Jul 2
FLAT: Revealing Hidden Latent-Conditioned Backdoor Failures in Federated Learning
arXiv:2508. 04064v2 Announce Type: replace-cross Abstract: Horizontal federated learning (HFL) backdoor audits often summarize model behavior through clean accuracy (CA), mean attack success rate (ASR), or a single known-trigger test.
arXiv Machine Learning
Jul 24
Finite-Sample Coverage Audits for High-Recall Candidate Generation: Certification and Learning-Theoretic Design
arXiv:2607. 21480v1 Announce Type: new Abstract: An initial high-recall stage in an empirical pipeline decides which items pass to later review, labelling, or modelling, and relevant items it misses are lost to every subsequent stage.
arXiv AI
Jul 17
Certified Domain Consistency for Multi-Domain Retrieval: Label-Free Per-Domain Contamination Control with Conformal Risk Guarantees
arXiv:2607. 14157v1 Announce Type: cross Abstract: Retrieval over corpora that mix several domains often returns relevant but wrong-domain evidence that ranking metrics miss and that conformal risk control bounds only marginally, under-covering the worst domains.
arXiv Machine Learning
Jul 10
False Confidence: Automated Labels Confound Fairness Audits in Cervical Spine Segmentation
arXiv:2607. 07852v1 Announce Type: cross Abstract: Automated segmentation of cervical-spine MRI is increasingly used in clinical workflows, yet no fairness audit exists for this anatomy.
arXiv AI
Jun 30
Sequential Fairness Auditing with Limited Output Access
arXiv:2606. 30338v1 Announce Type: new Abstract: External evaluations are becoming increasingly central to the governance of AI systems.
Hugging Face Trending Papers
Jun 15
Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget
Machine unlearning has been extensively studied in response to growing privacy concerns and regulatory requirements. However, auditing whether unlearning algorithms have truly erased the influence of specific data remains an open challenge.
arXiv Machine Learning
Aug 5
Evaluation Blindness: How Silent Measurement Failures Corrupt AI Systems from Training to Deployment
arXiv:2608. 02786v1 Announce Type: new Abstract: AI systems can fail silently.
arXiv Machine Learning
Jul 1
White-Box Sensitivity Auditing with Steering Vectors
arXiv:2601. 16398v3 Announce Type: replace-cross Abstract: Algorithmic audits are essential tools for examining systems for properties required by regulators or desired by operators.