arXiv Machine Learning

Lag Operator SSMs: A Geometric Framework for Structured State Space Modeling

arXiv:2512. 18965v2 Announce Type: replace Abstract: Structured State Space Models (SSMs), which are at the heart of the recently popular Mamba architecture, are powerful tools for sequence modeling.

arXiv Statistics ML
2d ago

Extending SSMs with the Exponentially Weighted Signature

arXiv:2603.19198v3 Announce Type: replace Abstract: We introduce the exponentially weighted signature (EWS), a continuous-time model that computes iterated integrals of a path, where each increment i...

By Alexandre Bloch, Benjamin Walker, Jo\"el Mouterde, Sam Morley, Samuel N. Cohen, Terry Lyons
arXiv Machine Learning
Aug 11

Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mamba

arXiv:2503. 18970v4 Announce Type: replace Abstract: Structured State Space Models (SSMs) have become a prominent class of sequence models, developed against two long-standing difficulties: the sequential computation and gradient propagation limits of Recurrent Neural Networks (RNNs), and the quadratic time and memory cost of self-attention in Transformers.

By Shriyank Somvanshi, Md Monzurul Islam, Mahmuda Sultana Mimi, Sazzad Bin Bashar Polock, Gaurab Chhetri, Anandi Dutta, Amir Rafe, Subasish Das
arXiv Machine Learning
Jul 28

Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functions

arXiv:2506. 05678v3 Announce Type: replace Abstract: The evolution of sequence modeling architectures, from recurrent neural networks and convolutional models to Transformers and structured state-space models, reflects ongoing efforts to address the diverse temporal dependencies inherent in sequential data.

By Haotian Jiang, Zeyu Bao, Shida Wang, Qianxiao Li
arXiv AI
2d ago

Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence

The paper proposes two extensions to State Space Models (SSMs) to reduce memory usage and improve performance. First, it introduces depth recurrence, allowing a looped SSM with fewer parameters to match the performance of a larger, non-recurrent model. Second, it advocates using a fixed time granularity across tasks by reshaping input sequences, which enhances how information is presented to the model. Both techniques consistently benefit four representative SSM architectures (LRU, S5, LinOSS, LrcSSM).

By M\'onika Farsang, Ramin Hasani, Daniela Rus, Radu Grosu
arXiv Machine Learning
Aug 14

A Simple State Space Model Excels at Multivariate Time Series Classification

arXiv:2605. 27406v2 Announce Type: replace Abstract: Structured state space models (SSMs) have recently emerged as a promising foundation for sequence modeling, with Mamba-based architectures demonstrating strong performance through input-dependent state transitions, albeit at considerable complexity.

By Hassan Saadatmand, Geoffrey I. Webb, Hamid Rezatofighi, Mahsa Salehi
arXiv Machine Learning
Aug 17

The Expressive Limits of Diagonal SSMs for State-Tracking

arXiv:2603. 01959v2 Announce Type: replace Abstract: State-Space Models (SSMs) have recently been shown to achieve strong empirical performance on a variety of long-range sequence modeling tasks while remaining efficient and highly-parallelizable.

By Mehran Shakerinava, Behnoush Khavari, Siamak Ravanbakhsh, Sarath Chandar