Hugging Face Trending Papers
The Price of Choice in Selective Prediction
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Oct 22
The Price of Choice in Selective Prediction
arXiv:2610. 29067v1 Announce Type: new Abstract: A selective classifier may be claimed to satisfy a risk-coverage contract if some rule from a predeclared menu (a threshold or a union of $K$ score intervals) attains coverage at least $\kappa$ and error at most $\alpha$.
arXiv AI
Oct 15
Certify the Decision, Not Every Score: Exact Top-k Selection from Few Evaluations
arXiv:2610.14996v1 Announce Type: new Abstract: Choosing a few robust algorithms or schedules can require evaluating every candidate under many adverse scenarios, even though an agent only needs to i...
arXiv Computation and Language
Sep 25
StepCOPS: Closed-Testing Lower-Tail Certificates for Language-Model Policy Selection
StepCOPS is a new method for selecting a language‑model policy from many checkpoints, prompts, and decoding rules by providing closed‑testing lower‑tail certificates. It uses an independent proposal split to nominate a lower‑tail floor for each candidate, applies exact binomial tests on a fresh certification split, and employs Holm’s step‑down procedure to certify a set of floors. In experiments across 24 configurations and 11 benchmarks, StepCOPS achieves 96.4% selected‑policy coverage, raises the certified floor by 1.5 points over prior methods, stays 0.6 points below a large‑reference jury oracle, and abstains in 2.4% of trials.
arXiv Machine Learning
Jun 9
A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Control
arXiv:2606. 08517v1 Announce Type: new Abstract: Selective predictors answer on confident inputs and abstain elsewhere; deploying one safely needs a single finite-sample certificate that simultaneously upper-bounds the selected risk, lower-bounds the acceptance probability $\pacc$ above a floor $\pmin$, and lower-bounds the deployment utility.
arXiv Machine Learning
Sep 17
How Many Labels Does Model Choice Need? Certificates and Budgets for Selective Prediction
arXiv:2609. 18622v1 Announce Type: new Abstract: Classifiers can make identical predictions yet require labels to compare their selective performance: confidence ranks weight the same errors differently.
Hugging Face Trending Papers
Sep 16
How Many Labels Does Model Choice Need? Certificates and Budgets for Selective Prediction
Classifiers can make identical predictions yet require labels to compare their selective performance: confidence ranks weight the same errors differently. We quantify this requirement for the area und...