arXiv Statistics ML
The geometry of AI validation: Exact certification limits for iid best-of-N search
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Sep 15
The geometry of AI validation: From structural blindness to reusable audits
The paper investigates how AI systems that perform best‑of‑n search require different validation strategies as the search width changes. It shows that auditing only small search widths leaves a gap in reliability estimates for larger widths, and proposes retaining candidate ranks and truth labels to estimate reliability across all widths up to N. The authors derive theoretical bounds on the minimax mean‑squared error, design procedures that achieve these bounds, and demonstrate that a shared audit can significantly reduce maximum error across many widths in practical CodeRM pools.
arXiv Machine Learning
Aug 4
When May a Model Replace the Experiment? Audits, Licenses, and the Price of Trust in Surrogate-Driven Design
arXiv:2608. 01378v1 Announce Type: new Abstract: Design campaigns in chemistry, materials science, and machine learning share a bottleneck: determining how good a candidate truly is requires an expensive evaluation - an experiment, a first-principles simulation, or a full training run.
arXiv AI
Oct 5
Information Limits of Low-Rank Approximation Certification
arXiv:2610.03321v1 Announce Type: cross Abstract: Low-rank approximation can require additional matrix--vector products to verify that its error meets a prescribed tolerance. We characterize this cer...
arXiv Statistics ML
Oct 27
Identities, Not Pairs: Distribution-Free False-Accept Control for Embedding-Based Verification
arXiv:2610.35277v1 Announce Type: new Abstract: Embedding-based verification often calibrates a false-accept threshold from all impostor pairs among $m$ identities, although these pairs share identit...
arXiv Machine Learning
Aug 11
The Cost of Adaptivity: Matching Lower Bounds Across Learning Problems
arXiv:2608. 08826v1 Announce Type: new Abstract: Adaptive procedures must work without nuisance information an oracle may use, such as a gradient scale or smoothness index, and robust procedures may have to answer queries whose coordinate and inspection time are chosen only after the data are seen.
arXiv Machine Learning
Sep 21
Available Guardrails: Certifying Selective Prediction across ML Systems
The paper introduces a method to certify selective prediction in machine learning systems by computing the availability of safety gates through exact-binomial inversion and dynamic programming. It demonstrates that a truth-informed planner can significantly improve mean coverage over naive approaches, and that reallocating error budgets further enhances coverage across diverse applications such as LLM tool‑calling, content moderation, lesion classification, and recommendation. The study highlights the importance of planning and finite‑sample estimation in ensuring reliable, granular deployment of selective predictors.