arXiv Statistics ML

Marginally Useful: An Information-Gap Identity in Conformal Prediction

arXiv Machine Learning
2d ago

Counterfactual Online Conformal Prediction Under Adaptive Logging

The paper addresses the failure of online conformal prediction when predictions influence actions that determine which outcomes are used for calibration. It introduces Propensity-Weighted Online Conformal Prediction (PW‑OCP), an inverse‑propensity‑weighted recursion that debiases calibration, and a doubly robust variant (DR‑OCP) that further reduces bias. Experiments on synthetic decision tasks, open bandit data, and financial rebalancing demonstrate that PW‑OCP and DR‑OCP improve counterfactual coverage and downstream regret while preserving prediction‑set sharpness.

By Xinyu Qiao, Yichen Lin, Kaihong Ji, Xue Wang, Tao Yao
arXiv Machine Learning
Aug 19

Dynamic Regime-Aware Conformal Calibration for Reliable Economic Forecast Intervals under Multiple Distribution Shifts

Dynamic Regime-Aware Conformal Prediction (DRACP) is a new method that blends density‑ratio estimation, localized kernel weighting, and probabilistic regime‑aware weighting with a self‑tuning online significance controller to produce reliable prediction intervals under multiple distribution shifts. The authors prove finite‑sample validity with oracle weights, provide a coverage‑gap bound for estimated weights, and give deterministic or regret guarantees for the online controller. In experiments on 48 real forecasting series—including euro‑area inflation, US macroeconomic and energy indicators, and daily financial data—DRACP achieves the most reliable calibration, maintaining coverage close to the nominal 0.90 and never falling below 0.80, while other methods achieve narrower intervals but with higher under‑coverage. whyItMatters":"DRACP offers a principled trade‑off between calibration and efficiency, ensuring that prediction intervals meet coverage standards even when economic data exhibit covariate shift, concept drift, and latent regimes."

By Bogdan Oancea
arXiv Statistics ML
Sep 1

Elements of Conformal Prediction

arXiv:2603.23923v2 Announce Type: replace-cross Abstract: Predictive inference is a fundamental task in statistics, traditionally addressed using parametric assumptions about the data distribution an...

By Matteo Sesia, Stefano Favaro
Hugging Face Trending Papers
Aug 11

Retrieval-Corrected Conformal Prediction for Time Series

Conformal prediction (CP) provides distribution-free prediction intervals for fixed forecasters, but its standard calibration procedure is often inefficient for time series data, where forecast errors are temporally dependent and change across time and operating conditions. Recent time series CP methods improve local calibration using recent, weighted, or localized residuals.

arXiv Machine Learning
Jun 3

Set-Preserving Calibration from Conformal P-Values to E-Values

arXiv:2606. 03600v1 Announce Type: cross Abstract: Standard conformal prediction (CP) procedures are typically formulated in terms of p-values, but reliance on p-values alone limits flexibility, for example, when combining dependent evidence across models or data splits.

By Nabil Alami, Jad Zakharia, Souhaib Ben Taieb
arXiv AI
Aug 12

Retrieval-Corrected Conformal Prediction for Time Series

arXiv:2608. 10553v1 Announce Type: cross Abstract: Conformal prediction (CP) provides distribution-free prediction intervals for fixed forecasters, but its standard calibration procedure is often inefficient for time series data, where forecast errors are temporally dependent and change across time and operating conditions.

By Sangjin Jin, Kangmin Kim, Junhyeong Lee, Yongjae Lee