arXiv Statistics ML
Identities, Not Pairs: Distribution-Free False-Accept Control for Embedding-Based Verification
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
arXiv Machine Learning
Jul 21
When Can You Debias an LLM Judge? Identifiability Limits, a Test, and Designs for Top-k Ranking
arXiv:2607. 02104v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as cheap, scalable judges that compare candidate outputs pairwise.
arXiv Machine Learning
Aug 18
Certifying Compressed Language Models: An Audit and a Statistical Toolkit
arXiv:2608. 15046v1 Announce Type: new Abstract: A fraction of a point of benchmark accuracy is the usual evidence that a compressed model is equivalent to its original.
arXiv Machine Learning
Oct 16
Sharp and Stable Label-Free Calibration Certificates from Logical Coherence
The paper presents a framework for deriving sharp, label‑free calibration certificates for query‑conditioned probabilistic predictors. It establishes minimax lower bounds on calibration residuals under various betting restrictions, shows that these bounds can be discontinuous and that Lipschitz stakes yield stable, sharp bounds. The authors also demonstrate practical algorithms—projected online gradient ascent and anytime‑valid betting—for generating fixed‑horizon and anytime‑valid certificates, and validate the approach on a preregistered study of four predictors across two datasets.
arXiv AI
Aug 11
Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits
arXiv:2608. 07913v1 Announce Type: cross Abstract: Selective-risk certificates promise that accepted outputs meet a declared error target.
arXiv AI
Jul 15
Sensitivity to Subjective Expected Utility Maximization: A Methodological Study, with an Illustrative Application to LLM Decision-Making
arXiv:2607. 11920v1 Announce Type: cross Abstract: Evaluating decisions made under uncertainty is hard when labeled outcomes are scarce, costly, or confounded with luck.
arXiv AI
Sep 24
Ask Which, Not How Good: Sizing Benchmarks Scored by an LLM
The study analyzes 373,019 judgments from LLM‑scored benchmarks, decomposing variance into system, item, judge, and interaction components via generalizability theory. It finds that with a single judge, generalizability converges to a ceiling determined by the system‑by‑judge variance, which is substantially lower in pairwise preference settings, allowing one judge to suffice. The research also reveals significant biases in presentation order and highlights that many published win‑rate claims fall below the measured floor of the benchmarks.