← Back to all news
arXiv AI By Jianqi Zhang, Yuchan Liu, Zeen Song, Yuefei Li, Fanjiang Xu

Fewer yet critical: Reducing Redundant Token Dependencies for Transformer-based Time Series Forecasting

Read the original on arXiv AI →

The paper introduces a token dependency selection strategy for Transformer-based time series forecasting. By jointly applying an attention entropy constraint and a prediction error constraint, the method identifies fewer but more critical inter-token dependencies, reducing the influence of redundant dependencies that can hurt generalization. Experiments on multiple datasets show that this approach improves forecasting performance across various Transformer models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 18

SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

SETTer is a transformer-based model designed for long‑term multivariate time‑series forecasting. It introduces decoupled self‑attention and hybrid masking to better handle high dimensionality and complex relationships, while adding explainable structures to highlight discriminative patterns. Experiments on real‑world benchmarks show that SETTer outperforms state‑of‑the‑art models in 88% of scenarios.

By Abraham Ezema, Chijioke Eze, Ferdinanda Ponci, Antonello Monti
llmsbenchmarks
More like this →
Hugging Face Trending Papers
Sep 17

SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

SETTer is a transformer-based model designed for long‑term multivariate time‑series forecasting. It introduces decoupled self‑attention and hybrid masking to better capture short‑ and long‑term patterns across time and channel dimensions, while adding simple explainable structures to highlight discriminative patterns. Experiments on real‑world benchmarks show that a single‑layer SETTer outperforms state‑of‑the‑art models in 88% of scenarios.

llmsbenchmarks
More like this →
arXiv Machine Learning
Jul 20

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

arXiv:2607. 15705v1 Announce Type: new Abstract: Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for demand-side management and energy management systems.

By Matthias Hertel, Sebastian P\"utz, Jonathan Kolar, Benjamin Sch\"afer, Ralf Mikut, Veit Hagenmeyer
llmsbenchmarks
More like this →