arXiv Machine Learning

Explicit Asymptotic Bounds for Sequential Calibration Beyond $T^{2/3}$

arXiv Machine Learning
Sep 17

Breaking the $T^{2/3}$ Barrier for Sequential Calibration

arXiv:2406. 13668v4 Announce Type: replace Abstract: A set of probabilistic forecasts is calibrated if each prediction of the forecaster closely approximates the empirical distribution of outcomes on the subset of timesteps where that prediction was made.

By Yuval Dagan, Constantinos Daskalakis, Maxwell Fishelson, Noah Golowich, Robert Kleinberg, Princewill Okoroafor
arXiv Machine Learning
Jul 23

Optimal Recalibration of an Online Predictor

arXiv:2607. 19689v1 Announce Type: cross Abstract: We study the problem of recalibrating an online predictor [KE17, OKS24]: given an arbitrary "hint" sequence of forecasts, the learner must output new predictions that are calibrated while incurring small excess error relative to the original forecasts, under a proper loss.

By Lunjia Hu, Kevin Tian, Chutong Yang
arXiv Machine Learning
Aug 24

Truthful Calibration Measures for Sequential Prediction

The paper investigates the feasibility of exact truthfulness in calibration measures for sequential binary prediction. It proves that exact truthfulness cannot coexist with completeness and soundness, even when outcomes are independent. The authors then provide two reductions that transform any base calibration measure into additively or multiplicatively approximately truthful ones, achieving a multiplicative truthfulness guarantee that improves upon previous results.

By Anagha Gokul, Jason Hartline, Lunjia Hu, Jonathan Ullman, Yifan Wu
arXiv Machine Learning
Jun 9

Proper Calibeating

arXiv:2605. 26703v2 Announce Type: replace-cross Abstract: The classic concept of "calibrated forecasts" and its more recent refinement, "calibeating," are defined with respect to the standard quadratic scoring rule.

By Dean P. Foster, Sergiu Hart
arXiv Machine Learning
Jun 18

Toward Simultaneously Optimal Regret in U-Calibration

arXiv:2606. 18527v1 Announce Type: cross Abstract: U-calibration studies online forecasting algorithms whose predictions can be consumed by any unknown downstream agent, guaranteeing sublinear regret simultaneously for all proper loss functions.

By Rafael Frongillo, Haipeng Luo, Nishant A. Mehta, Jon Schneider
arXiv Machine Learning
Oct 16

Computational and Statistical Hardness of Calibration Distance

The paper investigates the computational and statistical challenges of the calibration distance, a key metric for measuring miscalibration in probabilistic predictors. It presents an efficient algorithm for exact computation when the distribution has a uniform marginal and noiseless labels, and shows that the problem becomes NP‑hard if either assumption is dropped. The authors also provide a polynomial‑time approximation scheme for the general case and establish tight sample complexity bounds for estimating the calibration distance from data.

By Mingda Qiao