The paper critiques the prevalent use of mean squared error (MSE) for evaluating irregular time‑series forecasting, arguing that MSE is biased by timestamp sampling distributions. It introduces the Continuous‑time Squared Error (CSE), an importance‑weighted metric that theoretically offers a tighter asymptotic bound on continuous‑time risk than MSE. A comprehensive benchmark across synthetic, semi‑synthetic, and eight real‑world datasets demonstrates that CSE more accurately recovers continuous‑time risk, revealing limitations of relying solely on MSE.
By Rongwen Li, Haixin Xie, Xiao Wang, Changjian Chen
Existing research on irregular time-series forecasting has primarily focused on model design, while evaluation metrics remain insufficiently studied. Existing benchmarks typically use mean squared error (MSE) as the evaluation metric.
arXiv:2509. 26468v3 Announce Type: replace Abstract: Benchmark quality is critical for meaningful evaluation and sustained progress in time series forecasting, particularly with the rise of pretrained models.
By Oleksandr Shchur, Abdul Fatir Ansari, Caner Turkmen, Lorenzo Stella, Nick Erickson, Pablo Guerron, Michael Bohlke-Schneider, Yuyang Wang
arXiv:2503. 24007v4 Announce Type: replace-cross Abstract: In time series forecasting, covariates represent external factors that influence target variables.
By Yosuke Yamaguchi, Issei Suemitsu, Wenpeng Wei
arXiv:2609.13345v1 Announce Type: cross
Abstract: Probabilistic forecasting is central to decision-making under uncertainty, yet its methodological landscape has become increasingly fragmented across...
By Donia Besher, Rajdeep Pathak, Madhurima Panja, Tanujit Chakraborty
arXiv:2608. 14106v1 Announce Type: cross Abstract: When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking, as measured by cross-sectional correlation.
By Shu Wan, Miles Ma, Hank Zhu, Guangqi Liu, Stephen Wang, Qingsong Wen, Huan Liu
arXiv:2602. 12147v4 Announce Type: replace Abstract: Time series foundation models (TSFMs) are revolutionizing the forecasting landscape from specific dataset modeling to generalizable task evaluation.
By Zhongzheng Qiao, Sheng Pan, Anni Wang, Viktoriya Zhukova, Yong Liu, Xudong Jiang, Qingsong Wen, Mingsheng Long, Ming Jin, Chenghao Liu
WhatIf-Bench is a semi‑synthetic benchmark that tests how well time‑series forecasters use covariates to predict a target’s response to future events. It generates 37,665 tasks from mechanistic simulators of data centers, power grids, road traffic, and water networks, varying covariate numbers, historical events, and event length. Evaluation shows large‑language models outperform the best time‑series foundation models, with all models performing well on replayed events but struggling on novel covariate settings, highlighting generalization as the main challenge.
By Arjun Ashok, Mihir Parmar, Palash Goyal, Bhavana Dalvi Mishra, Chun-Liang Li, Irina Rish, Alexandre Drouin, Jinsung Yoon, Tomas Pfister
The paper introduces temporal predictive multiplicity, a framework that examines how models with nearly identical predictive performance can produce markedly different forecast trajectories in time‑series forecasting. It demonstrates that even when models meet the same performance criteria, they can diverge significantly across the entire forecast horizon, and that limiting multiplicity at individual horizons only partially mitigates this issue. Experiments on 19 neural forecasting architectures across 11 datasets confirm that trajectory‑level disagreement is largely independent of horizon‑wise disagreement, revealing a gap in prior multiplicity studies.
By Emanuele Albini, Francesca Toni, Saumitra Mishra, Francesco Leofante
arXiv:2607. 28124v1 Announce Type: new Abstract: As forecasts increasingly drive decisions in fields such as energy, transportation, and healthcare, understanding the historical data behind these predictions has become as crucial as the predictions themselves.
By Xu Zheng, Wei Cheng, Zhuomin Chen, Mo Sha, Jingchao Ni, Dongsheng Luo
arXiv:2602. 16224v2 Announce Type: replace Abstract: Time series data are prone to noise in various domains, and training samples may contain low-predictability patterns that deviate from the normal data distribution, leading to training instability or convergence to poor local minima.
By Xu Zhang, Peng Wang, Yichen Li, Wei Wang
Models with near-identical predictive performance can yield substantially different predictions, a phenomenon known as predictive multiplicity. Prior work has mostly studied this at the level of indiv...