arXiv Machine Learning

WhatIf-Bench: Benchmarking Intervention-Aware Time Series Forecasting

WhatIf-Bench is a semi‑synthetic benchmark that tests how well time‑series forecasters use covariates to predict a target’s response to future events. It generates 37,665 tasks from mechanistic simulators of data centers, power grids, road traffic, and water networks, varying covariate numbers, historical events, and event length. Evaluation shows large‑language models outperform the best time‑series foundation models, with all models performing well on replayed events but struggling on novel covariate settings, highlighting generalization as the main challenge.

arXiv AI
Oct 9

Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives

Timer‑M1 is a pretrained multivariate time‑series foundation model that learns with primitives for zero‑shot forecasting. It uses a primitive‑based data synthesis and pretraining pipeline to generate series with shared temporal and relational patterns, assembling them into multivariate samples with distinct channel roles. The model employs gated two‑dimensional Transformer blocks that dynamically allocate cross‑variate attention, achieving top performance on several large‑scale forecasting benchmarks.

By Haoran Zhang, Haixuan Liu, Xingjian Su, Yong Liu, Zhi Chen, Yuxuan Wang, Jianmin Wang, Mingsheng Long
arXiv Machine Learning
Oct 12

On the Encoding of Calendar Features in Covariate-Informed Time Series Foundation Models

The paper investigates how different calendar feature encodings influence the performance of covariate-informed time series foundation models (TSFMs). Seven encoding schemes—three target‑agnostic (ordinal, one‑hot, cyclic) and four target‑based—are systematically compared across four state‑of‑the‑art TSFMs on nine datasets that reflect social activity patterns such as weekends and holidays. Results show that encoding choice can change forecast accuracy by up to 9.9 % on non‑holidays and 83 % on holidays, with improvements mainly around holidays and varying effects across models.

By Alexander Kreusel, Maximilian Beichter, Alexandra Nikoltchovska, Benjamin Sch\"afer, Ralf Mikut, Frederik Zahn, Veit Hagenmeyer
arXiv AI
Oct 5

Time Series Forecasting Benchmarks Need Scenario-Grounded Stress Testing

The paper argues that current time‑series forecasting benchmarks are too narrow, rewarding low held‑out error while failing to capture real‑world failure modes such as structured input anomalies, regime shifts, and sensor faults. It proposes scenario‑grounded stress testing, where each test includes historical data, future targets, a semantic scenario, a failure operator, and a difficulty level, to make evaluation interpretable and deployment‑relevant.

By Yuyang Zhao, Lian Xu, Hao Xue