← Back to all news
arXiv Machine Learning By Qingwen Lin, Boyan Xu, Shifeng Xie, Zhifeng Hao, Keli Zhang, Ruichu Cai

HART: Hybrid Attention-RWKV Time series Foundation Model

Read the original on arXiv Machine Learning →

The paper introduces HART, a Hybrid Attention-RWKV Time-series Foundation Model that alternates attention and RWKV blocks to improve long-horizon forecasting. It identifies a problem in RWKV-7 called Placeholder Idling Decay, where repeated state transitions degrade historical information during non-autoregressive multi-step forecasting. Experiments on GIFT-Eval and TIME-Benchmark show that HART outperforms pure RWKV on medium and long horizons while matching short-horizon performance, and achieves comparable long-horizon MASE to a much larger Transformer with only 36 M parameters.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 22

A Hybrid Attention Model Learning Unified Time-aware Patch Representation for Irregular Multivariate Time Series Forecasting

The paper introduces a hybrid attention model that learns a unified time‑aware patch representation for irregular multivariate time series (IMTS) forecasting. It employs a time‑aware patch encoding to embed variable‑length intra‑patch timestamps, a time bias attention mechanism to adjust for temporal misalignment and asynchronous cross‑channel dependencies, and a hybrid causal mask on a decoder‑only Transformer to balance historical context with autoregressive forecasting. The authors also curate VersaTSA, a 30 B‑observation dataset preserving native sampling sparsity, and demonstrate state‑of‑the‑art zero‑shot performance on three IMTS benchmarks while remaining competitive on regular MTS tasks.

By Zhihao Lin, Li Lin, Qi Zhang, Kaiwen Xia, Shuai Wang, Jialin Qiao
llmsragefficiencybenchmarkssafety
More like this →
arXiv Machine Learning
6d ago

SACQ: Structured Decoding with Memory-Conditioned Refinement for Long-Horizon Forecasting

SACQ introduces a structured prediction head for long‑horizon time‑series forecasting that replaces the traditional flatten readout. It uses a two‑stage decoding process: a coarse patch‑grid scaffold is first generated, then each future position is refined via cross‑attention over historical memory, with a learned gate merging the corrections. A batch‑adaptive scaled log‑cosh loss further stabilizes training under long horizons and noisy labels, leading to superior MSE/MAE performance across multiple backbones with minimal overhead.

By Guo Cheng, Zhengzhuo Xu, Chenchen Jing, Jingyi Hou
benchmarkssafety
More like this →
arXiv AI
Sep 10

Memory in Deep Time-Series Models

arXiv:2609.06006v1 Announce Type: cross Abstract: Deep learning for time series has progressed through successive architectural paradigms, from recurrent networks and transformers to structured state...

By Minh Hoang Nguyen, Huu Hiep Nguyen, Manh Nguyen, Van Dai Do, Dung Nguyen, Hung Le
llmsragagents
More like this →
arXiv Machine Learning
Oct 5

Dual-Context Analog Retrieval for Time Series Forecasting

Dual-Context Analog Retrieval for Time Series Forecasting (DuoTS) introduces a two-stage forecasting approach that first generates a base forecast and then refines it patch by patch. Each refinement step fuses a current context, which focuses on recent tokens, with a detail context that incorporates retrieved analogs and their subsequent trajectories. Experiments on real-world datasets demonstrate that DuoTS achieves state‑of‑the‑art performance, and ablation studies confirm the importance of both contexts and the refinement mechanism.

By Jung Min Choi, Ngoc Son Le, Ibram Abdelmalak, Vijaya Krishna Yalavarthi, Lars Schmidt-Thieme
benchmarks
More like this →