arXiv Machine Learning
The Price of Choice in Selective Prediction
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Oct 15
Certify the Decision, Not Every Score: Exact Top-k Selection from Few Evaluations
arXiv:2610.14996v1 Announce Type: new Abstract: Choosing a few robust algorithms or schedules can require evaluating every candidate under many adverse scenarios, even though an agent only needs to i...
arXiv Machine Learning
Jun 9
A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Control
arXiv:2606. 08517v1 Announce Type: new Abstract: Selective predictors answer on confident inputs and abstain elsewhere; deploying one safely needs a single finite-sample certificate that simultaneously upper-bounds the selected risk, lower-bounds the acceptance probability $\pacc$ above a floor $\pmin$, and lower-bounds the deployment utility.
arXiv Machine Learning
Sep 17
How Many Labels Does Model Choice Need? Certificates and Budgets for Selective Prediction
arXiv:2609. 18622v1 Announce Type: new Abstract: Classifiers can make identical predictions yet require labels to compare their selective performance: confidence ranks weight the same errors differently.
arXiv Machine Learning
Jun 16
When Does q-error Predict Plan Regret? Three Regimes of Cardinality-Estimation Error
arXiv:2606. 15600v1 Announce Type: cross Abstract: Cardinality-estimation (CE) research ranks estimators by q-error, yet it is well known that q-error is an imperfect proxy for query-plan quality.
arXiv Machine Learning
Sep 21
Available Guardrails: Certifying Selective Prediction across ML Systems
The paper introduces a method to certify selective prediction in machine learning systems by computing the availability of safety gates through exact-binomial inversion and dynamic programming. It demonstrates that a truth-informed planner can significantly improve mean coverage over naive approaches, and that reallocating error budgets further enhances coverage across diverse applications such as LLM tool‑calling, content moderation, lesion classification, and recommendation. The study highlights the importance of planning and finite‑sample estimation in ensuring reliable, granular deployment of selective predictors.
arXiv Machine Learning
Sep 1
PruneShift: A Framework for Evaluating Decision Reliability in Structured Pruning
arXiv:2608.29765v1 Announce Type: new Abstract: Structured pruning uses surrogate objectives because direct task evaluation over every feasible mask is too expensive. Most evaluations report average...