arXiv Machine Learning
Jul 3
Extreme Adaptive Transformer for Time Series Forecasting
arXiv:2607. 02437v1 Announce Type: new Abstract: Time series forecasting remains challenging when the underlying data contain rare but critical extreme events.
FANTF: Fuzzy attention network based transformers for time series analysis
The paper presents FANTF, a Fuzzy Attention Network-Based Transformer that integrates fuzzy logic into transformer architectures via a novel fuzzy attention mechanism inspired by Type‑2 fuzzy set theory. This mechanism employs a learnable Gaussian uncertainty perturbation to model uncertainty in attention relevance scores, producing softer, more diverse attention distributions and uncertainty‑aware interpretations. Experiments across five transformer backbones on real‑life datasets show statistically significant improvements in forecasting, classification, and anomaly detection, especially under noisy and non‑stationary conditions.
Related stories
arXiv AI
Sep 10
Fewer yet critical: Reducing Redundant Token Dependencies for Transformer-based Time Series Forecasting
The paper introduces a token dependency selection strategy for Transformer-based time series forecasting. By jointly applying an attention entropy constraint and a prediction error constraint, the method identifies fewer but more critical inter-token dependencies, reducing the influence of redundant dependencies that can hurt generalization. Experiments on multiple datasets show that this approach improves forecasting performance across various Transformer models.
arXiv Machine Learning
Sep 18
SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting
SETTer is a transformer-based model designed for long‑term multivariate time‑series forecasting. It introduces decoupled self‑attention and hybrid masking to better handle high dimensionality and complex relationships, while adding explainable structures to highlight discriminative patterns. Experiments on real‑world benchmarks show that SETTer outperforms state‑of‑the‑art models in 88% of scenarios.
arXiv AI
Jun 26
PMDformer: Patch-Mean Decoupling Information Transformer for Long-term Forecasting
arXiv:2606. 26549v1 Announce Type: new Abstract: Long-term time series forecasting (LTSF) plays a crucial role in fields such as energy management, finance, and traffic prediction.
arXiv Machine Learning
Sep 21
A Lightweight Plug-in Gate for Transformer-Based Time-Series Forecasters
The paper introduces a lightweight pre‑encoder gate for Transformer‑based time‑series forecasters, which assigns sigmoid scores to covariate representations before they enter the encoder. The gate is evaluated as a plug‑in for models such as TimeXer, iTransformer, and PatchTST on datasets including ETTm1, ETTm2, Traffic, Energy, and ILI, showing competitive performance and the ability to regulate covariate admission via a usage penalty. Experiments also explore gate placement, initialization, and feature importance using VIF‑informed permutation diagnostics.
Hugging Face Trending Papers
Sep 17
SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting
SETTer is a transformer-based model designed for long‑term multivariate time‑series forecasting. It introduces decoupled self‑attention and hybrid masking to better capture short‑ and long‑term patterns across time and channel dimensions, while adding simple explainable structures to highlight discriminative patterns. Experiments on real‑world benchmarks show that a single‑layer SETTer outperforms state‑of‑the‑art models in 88% of scenarios.
arXiv AI
Jul 3
Self-Gating Attention for Efficient Time Series Forecasting
arXiv:2607. 02344v1 Announce Type: cross Abstract: Transformer architectures have shown strong potential in time series forecasting, where multi-head self-attention is widely used to capture temporal dependencies across historical timestamps.
arXiv Machine Learning
Jun 5
FATE: Focal-modulated Attention Encoder for Multivariate Time-series Forecasting
arXiv:2408. 11336v3 Announce Type: replace Abstract: Climate change stands as one of the most pressing global challenges of the twenty-first century, with far-reaching consequences such as rising sea levels, melting glaciers, and increasingly extreme weather patterns.
arXiv AI
Jun 2
Generic Interpretation Approach for Transformer Models Incorporating Heterogenous Attention Structures
arXiv:2605. 27458v2 Announce Type: replace-cross Abstract: Transformer has significantly propelled the development of artificial intelligence, and certainly the development of agents as well.
arXiv Machine Learning
Jun 19
PaAno+: Multiscale Encoding and Cross-Variable Attention for Time Series Anomaly Detection
arXiv:2606. 20055v1 Announce Type: new Abstract: Time-series anomaly detection has significant practical value for industrial and medical monitoring, as well as other critical domains.
arXiv Machine Learning
Jul 20
A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods
arXiv:2607. 15705v1 Announce Type: new Abstract: Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for demand-side management and energy management systems.
arXiv Machine Learning
Oct 15
Beyond Static Positional Encoding: Bayesian Time-aware Attention for Non-Stationary Long-term Time Series Forecasting
arXiv:2610.19603v1 Announce Type: new Abstract: Transformers have demonstrated strong performance in long-term time series forecasting (LTSF), yet modeling non-stationary temporal dependencies remain...