arXiv:2603. 01959v2 Announce Type: replace Abstract: State-Space Models (SSMs) have recently been shown to achieve strong empirical performance on a variety of long-range sequence modeling tasks while remaining efficient and highly-parallelizable.
By Mehran Shakerinava, Behnoush Khavari, Siamak Ravanbakhsh, Sarath Chandar
arXiv:2607. 21000v1 Announce Type: new Abstract: Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and active overwriting of stale ones.
By Hyuk Lim, Seunghyun Yoon
arXiv:2608. 03425v1 Announce Type: new Abstract: Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms.
By Xiaohe Li, Yang Lu
arXiv:2607. 10110v1 Announce Type: new Abstract: Recent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than from additional independent parameters.
By Zhenxuan Yu, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa
Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and active overwriting of stale ones. In our diagnostic suite, the strongest efficient baselines tend to solve only one side well.
arXiv:2602. 14814v3 Announce Type: replace Abstract: Over the last years, state-tracking tasks, particularly permutation composition, have become a testbed to understand the limits of sequence models architectures like Transformers and RNNs (linear and non-linear).
By Julien Siems, Riccardo Grazzi, Korbinian P\"oppel, Kirill Kalinin, Hitesh Ballani, Babak Rahmani