arXiv Machine Learning

A Hybrid Attention Model Learning Unified Time-aware Patch Representation for Irregular Multivariate Time Series Forecasting

The paper introduces a hybrid attention model that learns a unified time‑aware patch representation for irregular multivariate time series (IMTS) forecasting. It employs a time‑aware patch encoding to embed variable‑length intra‑patch timestamps, a time bias attention mechanism to adjust for temporal misalignment and asynchronous cross‑channel dependencies, and a hybrid causal mask on a decoder‑only Transformer to balance historical context with autoregressive forecasting. The authors also curate VersaTSA, a 30 B‑observation dataset preserving native sampling sparsity, and demonstrate state‑of‑the‑art zero‑shot performance on three IMTS benchmarks while remaining competitive on regular MTS tasks.

arXiv AI
Oct 9

Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives

Timer‑M1 is a pretrained multivariate time‑series foundation model that learns with primitives for zero‑shot forecasting. It uses a primitive‑based data synthesis and pretraining pipeline to generate series with shared temporal and relational patterns, assembling them into multivariate samples with distinct channel roles. The model employs gated two‑dimensional Transformer blocks that dynamically allocate cross‑variate attention, achieving top performance on several large‑scale forecasting benchmarks.

By Haoran Zhang, Haixuan Liu, Xingjian Su, Yong Liu, Zhi Chen, Yuxuan Wang, Jianmin Wang, Mingsheng Long
arXiv Machine Learning
6d ago

The Unused Input: The Initial State as a Conditioning Interface for Recurrent TSFMs

The paper investigates whether the initial recurrent state in time‑series foundation models (TSFMs) can be leveraged to improve forecasting accuracy. Using the xLSTM‑based zero‑shot forecaster TiRex, the authors freeze the backbone and train a small head that writes a context vector into the initial state, demonstrating modest but consistent gains across several datasets. The study shows that the initial state can serve both as a per‑dataset adaptation surface and as a channel for exogenous information, suggesting a new avenue for enhancing recurrent TSFMs.

By Bernhard Voggenberger, Christian Ganh\"or, Levente Z\'olyomi, Sebastian B\"ock, Sepp Hochreiter
arXiv Machine Learning
Oct 5

Dual-Context Analog Retrieval for Time Series Forecasting

Dual-Context Analog Retrieval for Time Series Forecasting (DuoTS) introduces a two-stage forecasting approach that first generates a base forecast and then refines it patch by patch. Each refinement step fuses a current context, which focuses on recent tokens, with a detail context that incorporates retrieved analogs and their subsequent trajectories. Experiments on real-world datasets demonstrate that DuoTS achieves state‑of‑the‑art performance, and ablation studies confirm the importance of both contexts and the refinement mechanism.

By Jung Min Choi, Ngoc Son Le, Ibram Abdelmalak, Vijaya Krishna Yalavarthi, Lars Schmidt-Thieme
arXiv AI
Aug 25

NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting

NeST is a framework that adapts large language models (LLMs) for continuous time‑series forecasting by creating neighborhood‑aware text prototypes and aligning them with temporal representations through a nearest‑neighbor contrastive objective. It retrieves the most relevant prototypes and uses them to conditionally modulate time‑series features, enabling more effective integration of textual and temporal information. Experiments show that NeST outperforms state‑of‑the‑art methods on eight benchmarks, reduces MSE by 1.2% for long‑term forecasting, improves zero‑shot forecasting by 4.9%, and boosts R² by 3.3% on a real‑world photovoltaic power forecasting task.

By Jayanie Bogahawatte, Sachith Seneviratne, Maneesha Perera, Saman Halgamuge
arXiv Machine Learning
Oct 12

HART: Hybrid Attention-RWKV Time series Foundation Model

The paper introduces HART, a Hybrid Attention-RWKV Time-series Foundation Model that alternates attention and RWKV blocks to improve long-horizon forecasting. It identifies a problem in RWKV-7 called Placeholder Idling Decay, where repeated state transitions degrade historical information during non-autoregressive multi-step forecasting. Experiments on GIFT-Eval and TIME-Benchmark show that HART outperforms pure RWKV on medium and long horizons while matching short-horizon performance, and achieves comparable long-horizon MASE to a much larger Transformer with only 36 M parameters.

By Qingwen Lin, Boyan Xu, Shifeng Xie, Zhifeng Hao, Keli Zhang, Ruichu Cai
arXiv Machine Learning
Oct 7

QiYao-I: A Manifold Based Foundation Model for Irregular Multivariate Time Series Forecasting

QiYao-I is a manifold‑based foundation model designed for irregular multivariate time series forecasting. It introduces a sampling‑conditioned temporal manifold attention mechanism that maps real timestamps into a learnable manifold feature space, and a dynamic variable interaction mechanism that selectively passes messages across variables while respecting asynchronous observations. Experiments on real‑world benchmarks show that QiYao‑I outperforms existing foundation and end‑to‑end irregular forecasting models, especially in zero‑shot and few‑shot scenarios.

By Linfeng Wang, Ruitong Zhang, Kai Zhao, Yang Shu, Zhongwen Rao, Meng Wang, Yijie Li, Bin Yang, Chenjun Guo