The paper presents a framework for deriving sharp, label‑free calibration certificates for query‑conditioned probabilistic predictors. It establishes minimax lower bounds on calibration residuals under various betting restrictions, shows that these bounds can be discontinuous and that Lipschitz stakes yield stable, sharp bounds. The authors also demonstrate practical algorithms—projected online gradient ascent and anytime‑valid betting—for generating fixed‑horizon and anytime‑valid certificates, and validate the approach on a preregistered study of four predictors across two datasets.
arXiv:2610.35277v1 Announce Type: new Abstract: Embedding-based verification often calibrates a false-accept threshold from all impostor pairs among $m$ identities, although these pairs share identit...
arXiv:2609.38917v1 Announce Type: new Abstract: A classifier's conditional accuracy can change while its confidence distribution stays exactly the same. We study the worst-case movement of the reliab...
arXiv:2610.09541v1 Announce Type: cross Abstract: Signals that predict whether a chain-of-thought (CoT) trace is correct are compared by AUC, but deploying one requires a threshold with a guarantee....
arXiv:2608. 07913v1 Announce Type: cross Abstract: Selective-risk certificates promise that accepted outputs meet a declared error target.
The paper examines how an agent’s probability report is evaluated twice—once by a strictly proper scoring rule and again by an approval rule that determines a decision. It shows that when the approval rule is welfare‑maximizing, it cannot be affine, yet the resulting distortion is predictable and can be mitigated by a reserve report that neutralizes the cost of pretending to be the marginal type. A Lipschitz rule with a single kink achieves first‑best welfare, while smooth rules cannot, and the key constraint is the steepness of the rule rather than its smoothness.
Selective-risk certificates promise that accepted outputs meet a declared error target. We develop Fed-SRC, a score-agnostic certificate for federated, differentially private, adaptively monitored retrieval-augmented generation.
arXiv:2606. 08517v1 Announce Type: new Abstract: Selective predictors answer on confident inputs and abstain elsewhere; deploying one safely needs a single finite-sample certificate that simultaneously upper-bounds the selected risk, lower-bounds the acceptance probability $\pacc$ above a floor $\pmin$, and lower-bounds the deployment utility.
arXiv:2502. 15131v4 Announce Type: replace-cross Abstract: We study the fundamental problem of calibrating a linear binary classifier of the form $\sigma(\hat{w}^\top x)$, where the feature vector $x$ is Gaussian, $\sigma$ is a link function, and $\hat{w}$ is an estimator of the true linear weight $w^\star$.
The paper introduces a distribution‑free certification layer that can be applied to any crash‑severity prediction model without modifying the model itself. It provides guarantees for ordinal outcomes, per‑class validity, transfer of coverage to unobserved severities, and one‑sided certificates under deployment shift, all grounded in a functional of the true data law. The framework is evaluated on 5.2 million Texas records, demonstrating a model‑independent lower bound on set width for vulnerable road users and is released as an open‑source package with theorem‑level tests.
arXiv:2609.39123v1 Announce Type: new Abstract: When optimizing an expensive black-box function sequentially, as in hyperparameter optimization, we may want to stop once the best evaluated value is c...
arXiv:2609.00691v1 Announce Type: new Abstract: Post-hoc out-of-distribution detectors are fitted on a finite reference set, so every score they produce is an estimate. If we had chosen a different s...
The paper derives precise cost formulas for self‑calibrating monitors that adjust thresholds online to maintain a specified long‑run false‑alarm rate under arbitrary drift. It shows that the guarantee is an accounting identity, independent of the monitored signal, and provides exact evidence identities for both step and ramp drift scenarios, as well as an exact law for the fluctuation of the certificate’s own alarm rate. Additionally, it proves that any monitor designed to tolerate a drift class is blind to all faults in the difference of that class, identifying the blind set for speed‑bounded drift classes and quantifying power outside this set with a sharp Gaussian projection bound.