Yes, Transformers are Effective for Time Series Forecasting (+ Autoformer)

Probabilistic Time Series Forecasting with 🤗 Transformers

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

arXiv:2607. 15705v1 Announce Type: new Abstract: Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for demand-side management and energy management systems.

Fewer yet critical: Reducing Redundant Token Dependencies for Transformer-based Time Series Forecasting

The paper introduces a token dependency selection strategy for Transformer-based time series forecasting. By jointly applying an attention entropy constraint and a prediction error constraint, the method identifies fewer but more critical inter-token dependencies, reducing the influence of redundant dependencies that can hurt generalization. Experiments on multiple datasets show that this approach improves forecasting performance across various Transformer models.

Why Transformers Need Positional Encoding For Time Series: A Visual Guide

The article explains how transformers, which rely on self‑attention, can lose the natural order of time‑series data when fed scalar observations. It discusses the role of positional encoding in re‑introducing sequence order and provides a visual guide to illustrate this concept.

Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

arXiv:2607. 22299v1 Announce Type: cross Abstract: Forecasting multiple time-series with high-dimensional covariates presents a core challenge: unifying common temporal patterns while retaining meaningful series-specific information.

Introducing Optimum: The Optimization Toolkit for Transformers at Scale

AROpt: An Optimization Method for Autoregressive Time Series Forecasting

arXiv:2602. 02288v3 Announce Type: replace Abstract: Current time-series forecasting models are primarily based on transformer-style neural networks.

Timm ❤️ Transformers: Use any timm model with transformers

Extreme Adaptive Transformer for Time Series Forecasting

arXiv:2607. 02437v1 Announce Type: new Abstract: Time series forecasting remains challenging when the underlying data contain rare but critical extreme events.

Patch Time Series Transformer in Hugging Face

A Lightweight Plug-in Gate for Transformer-Based Time-Series Forecasters

The paper introduces a lightweight pre‑encoder gate for Transformer‑based time‑series forecasters, which assigns sigmoid scores to covariate representations before they enter the encoder. The gate is evaluated as a plug‑in for models such as TimeXer, iTransformer, and PatchTST on datasets including ETTm1, ETTm2, Traffic, Energy, and ILI, showing competitive performance and the ability to regulate covariate admission via a usage penalty. Experiments also explore gate placement, initialization, and feature importance using VIF‑informed permutation diagnostics.

Mixture of Experts (MoEs) in Transformers