The paper critiques the prevalent use of mean squared error (MSE) for evaluating irregular time‑series forecasting, arguing that MSE is biased by timestamp sampling distributions. It introduces the Continuous‑time Squared Error (CSE), an importance‑weighted metric that theoretically offers a tighter asymptotic bound on continuous‑time risk than MSE. A comprehensive benchmark across synthetic, semi‑synthetic, and eight real‑world datasets demonstrates that CSE more accurately recovers continuous‑time risk, revealing limitations of relying solely on MSE.
By Rongwen Li, Haixin Xie, Xiao Wang, Changjian Chen
Existing research on irregular time-series forecasting has primarily focused on model design, while evaluation metrics remain insufficiently studied. Existing benchmarks typically use mean squared error (MSE) as the evaluation metric.
arXiv:2609.16415v1 Announce Type: cross
Abstract: Pedestrian-count forecasting supports pedestrian-oriented Intelligent Transportation Systems (ITS), including crowd monitoring, pedestrian-traffic st...
By Theivaprakasham Hari, Ziteng Li, Yanan Xin, Winnie Daamen, Serge Hoogendoorn
arXiv:2602. 16224v2 Announce Type: replace Abstract: Time series data are prone to noise in various domains, and training samples may contain low-predictability patterns that deviate from the normal data distribution, leading to training instability or convergence to poor local minima.
By Xu Zhang, Peng Wang, Yichen Li, Wei Wang
arXiv:2606. 18367v1 Announce Type: new Abstract: Standard benchmarks evaluate time series foundation models (TSFMs) using aggregate metrics, but these can mask severe failures in critical operating regimes.
By Yingshuo Wang, Xian Sun, Lingdong Kong, Wei Gao, Yanhang Li, Zhichao Fan, Zexin Zhuang
The paper introduces Generalized Gibbs Ensemble Weighting (GGEW), a probabilistic framework that assigns weights to forecasting models using a Gibbs-style exponential transformation of normalized predictive loss. GGEW extends basic weighting through numerical stabilization, diversity-aware score corrections, and online hyperparameter adaptation, yielding variants such as Stable Gibbs weighting, Directional Gibbs-NCL, and Symmetric Gibbs-NCL. The authors evaluate GGEW on M4 competition submissions and real-world datasets (Monash Traffic, Electricity, Solar), finding that Gibbs-style adaptive weighting is competitive across various settings, though performance varies by dataset, horizon, and deployment protocol.
By Prasen R. Nuthanakaluva, Nava K. Gaddam