arXiv Machine Learning

Leave a Window Out: Modifying the Jackknife for Predictive Inference in Time Series

arXiv:2605. 30292v2 Announce Type: replace-cross Abstract: Conformal prediction methods enjoy strong theoretical and empirical predictive inference performance, provided the data is exchangeable and is treated symmetrically during training.

arXiv Machine Learning
Sep 24

Rolling Conformal Prediction in Sequential Model Training

Rolling Conformal Prediction (rolling‑CP) is a distribution‑free predictive inference method designed for sequential model training. It calibrates each incoming observation against the current predictor and incorporates it into future training, eliminating the need for data splitting. For exchangeable data, rolling‑CP guarantees marginal coverage with a universal factor‑two bound, and for i.i.d. streams it provides high‑probability training‑conditional validity over time, improving to the target level under stability conditions.

By Chen Cheng, Ruiting Liang, Rina Foygel Barber
arXiv AI
Aug 19

SPACE: Sample-cloud Predictive Adaptive Conformal Ellipsoids for Multivariate Time-Series Forecasting

SPACE is a conformal wrapper that creates ellipsoidal joint prediction regions for multivariate time‑series forecasts by estimating time‑local covariance directly from the current forecast sample cloud. It calibrates the region’s radius using a dynamic backward window‑selection scheme, avoiding reliance on historical residuals. Experiments on diverse datasets show that SPACE improves joint and rolling coverage, achieving better coverage‑efficiency tradeoffs than existing wrappers.

By Baishi Li, Kelvin J. L. Koa, Ke-Wei Huang
arXiv AI
Aug 19

Beyond MSE: Rethinking the Evaluation Metric and Benchmarking for Irregular Time Series Forecasting

The paper critiques the prevalent use of mean squared error (MSE) for evaluating irregular time‑series forecasting, arguing that MSE is biased by timestamp sampling distributions. It introduces the Continuous‑time Squared Error (CSE), an importance‑weighted metric that theoretically offers a tighter asymptotic bound on continuous‑time risk than MSE. A comprehensive benchmark across synthetic, semi‑synthetic, and eight real‑world datasets demonstrates that CSE more accurately recovers continuous‑time risk, revealing limitations of relying solely on MSE.

By Rongwen Li, Haixin Xie, Xiao Wang, Changjian Chen
arXiv Machine Learning
Aug 19

Dynamic Regime-Aware Conformal Calibration for Reliable Economic Forecast Intervals under Multiple Distribution Shifts

Dynamic Regime-Aware Conformal Prediction (DRACP) is a new method that blends density‑ratio estimation, localized kernel weighting, and probabilistic regime‑aware weighting with a self‑tuning online significance controller to produce reliable prediction intervals under multiple distribution shifts. The authors prove finite‑sample validity with oracle weights, provide a coverage‑gap bound for estimated weights, and give deterministic or regret guarantees for the online controller. In experiments on 48 real forecasting series—including euro‑area inflation, US macroeconomic and energy indicators, and daily financial data—DRACP achieves the most reliable calibration, maintaining coverage close to the nominal 0.90 and never falling below 0.80, while other methods achieve narrower intervals but with higher under‑coverage. whyItMatters":"DRACP offers a principled trade‑off between calibration and efficiency, ensuring that prediction intervals meet coverage standards even when economic data exhibit covariate shift, concept drift, and latent regimes."

By Bogdan Oancea
arXiv Machine Learning
Jul 1

On Optimal Data Splitting for Split Conformal Prediction

arXiv:2606. 31600v1 Announce Type: cross Abstract: Conformal prediction and its variants, including the split conformal prediction, provide a distribution-free framework for uncertainty quantification by constructing prediction intervals or sets with finite-sample coverage guarantees.

By Sayan Das, Bahram Yaghooti, Todd A. Kuffner, Soumendra N. Lahiri
arXiv AI
Aug 12

Retrieval-Corrected Conformal Prediction for Time Series

arXiv:2608. 10553v1 Announce Type: cross Abstract: Conformal prediction (CP) provides distribution-free prediction intervals for fixed forecasters, but its standard calibration procedure is often inefficient for time series data, where forecast errors are temporally dependent and change across time and operating conditions.

By Sangjin Jin, Kangmin Kim, Junhyeong Lee, Yongjae Lee