arXiv AI
Efficient Linear Attention for Multivariate Time Series Modeling via Entropy Calibration
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 3
Self-Gating Attention for Efficient Time Series Forecasting
arXiv:2607. 02344v1 Announce Type: cross Abstract: Transformer architectures have shown strong potential in time series forecasting, where multi-head self-attention is widely used to capture temporal dependencies across historical timestamps.
arXiv Machine Learning
Oct 7
Retrieval Is Not Enough: Refreshing Memory for Frozen Time-Series Forecasters
arXiv:2610.07834v1 Announce Type: new Abstract: Retrieval-augmented time-series forecasting uses the continuations of historical segments similar to the current context as references for a forecaster...
arXiv AI
Jun 9
InA-Probe: Instruction-Aware Active Probing for Time Series Forecasting with LLMs
arXiv:2606. 08601v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently demonstrated impressive potential for time series forecasting.
arXiv AI
Jun 10
CITRAS: Covariate-Informed Transformer for Time Series Forecasting
arXiv:2503. 24007v4 Announce Type: replace-cross Abstract: In time series forecasting, covariates represent external factors that influence target variables.
arXiv AI
Oct 12
Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels
The paper introduces Outlier-Aware Attention Linearization (OAL), a method that improves linearized softmax attention by explicitly accounting for a few outlier dimensions that disproportionately affect query-key dot products. OAL builds on a second-order Taylor approximation, uses a Hadamard-product formulation to expose dimension-specific contributions, and groups dimensions with similar scales to share coefficients while treating outliers separately. Experiments on Qwen2.5-0.5B show an 8.9% reduction in frozen-attention output MSE, and fine-tuning with LoRA lowers WikiText-103 perplexity from 31.71 to 27.50 while boosting performance on PIQA, HellaSwag, and ARC-E.
arXiv Machine Learning
Oct 28
GateTS: Versatile and Efficient Forecasting via Attention-Inspired routed Mixture-of-Experts
GateTS introduces a streamlined univariate forecasting model that merges sparse Mixture-of-Experts computation with an attention-inspired gating mechanism, eliminating the need for auxiliary load‑balancing losses. The architecture achieves superior predictive accuracy across both short‑ and long‑term horizons, including intermittent signals, while using far fewer parameters than leading transformer models such as PatchTST. Empirical tests on diverse datasets show GateTS is computationally more efficient than LSTM, enabling cost‑effective inference for real‑world time‑series applications.