arXiv AI August 5, 2026 By Xiuding Cai, Xueyao Wang, Yaoyao Zhu, Yu Yao

MambaTS: Improved Selective State Space Models for Long-term Time Series Forecasting

arXiv:2405. 16440v2 Announce Type: replace-cross Abstract: In recent years, Transformers have become the de-facto architecture for long-term time series forecasting (LTSF), yet they face challenges associated with the self-attention mechanism, including quadratic complexity and permutation-invariant bias.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jun 4

MesaNet: Sequence Modeling by Locally Optimal Test-Time Training

arXiv:2506. 05233v2 Announce Type: replace-cross Abstract: Sequence modeling is currently dominated by causal transformer architectures that use softmax self-attention.

By Johannes von Oswald, Nino Scherrer, Seijin Kobayashi, Luca Versari, Songlin Yang, Sarthak Mittal, Maximilian Schlegel, Kaitlin Maile, Yanick Schimpf, Oliver Sieberling, Alexander Meulemans, Rif A. Saurous, Guillaume Lajoie, Charlotte Frenkel, Razvan Pascanu, Blaise Ag\"uera y Arcas, Jo\~ao Sacramento

llms benchmarks

arXiv Machine Learning

Jul 2

TiRex-2: Generalizing TiRex to Multivariate Data and Streaming

arXiv:2607. 01204v1 Announce Type: new Abstract: We introduce TiRex-2, a recurrent xLSTM-based time series foundation model that generalizes the univariate TiRex to multivariate forecasting with both past and future covariates.

By Patrick Podest, Marco Pichler, Elias B\"urger, Levente Z\'olyomi, Bernhard Voggenberger, Wilhelm Berghammer, Daniel Klotz, Sebastian B\"ock, G\"unter Klambauer, Sepp Hochreiter

llms benchmarks

arXiv Machine Learning

Jun 11

On Subquadratic Architectures: From Applications to Principles

arXiv:2606. 12364v1 Announce Type: new Abstract: Transformers dominate modern sequence modeling, but their quadratic attention incurs substantial computational cost.

By Anamaria-Roberta Hartl, Levente Z\'olyomi, David Stap, Pieter-Jan Hoedt, Niklas Schmidinger, Lukas Hauzenberger, Sebastian B\"ock, G\"unter Klambauer, Sepp Hochreiter

llms efficiency

arXiv AI

Jun 17

FoundCause: Causal Discovery with Latent Confounders from Observational Data

arXiv:2606. 17516v1 Announce Type: cross Abstract: Causal discovery from observational data remains challenging due to the need to recover directed structure and latent confounding without interventions.

By Patrick Bl\"obaum, Krishnakumar Balasubramanian, Shiva Prasad Kasiviswanathan

llms

arXiv AI

2d ago

OTIS: Learning High-Quality Time Series Features With Tiny Encoders

arXiv:2410. 07299v3 Announce Type: replace-cross Abstract: We introduce OTIS, an open time series encoder that yields high-quality time series features for downstream deployment on any system, including resource-constrained wearables and industrial sensors.

By \"Ozg\"un Turgut, Philip M\"uller, Martin J. Menten, Daniel Rueckert

benchmarks

arXiv Machine Learning

Jul 30

Dense Local Dependencies Induce Attention-Logit Explosion and Training Instability During Long-Sequence Transformer Training

arXiv:2505. 15548v2 Announce Type: replace Abstract: Autoregressive transformer language models frequently exhibit training instability when trained on long sequences, particularly under low-precision arithmetic.

By Suvadeep Hajra

llms

Related stories

MesaNet: Sequence Modeling by Locally Optimal Test-Time Training

TiRex-2: Generalizing TiRex to Multivariate Data and Streaming

On Subquadratic Architectures: From Applications to Principles

FoundCause: Causal Discovery with Latent Confounders from Observational Data

OTIS: Learning High-Quality Time Series Features With Tiny Encoders

Dense Local Dependencies Induce Attention-Logit Explosion and Training Instability During Long-Sequence Transformer Training