arXiv:2603. 03612v3 Announce Type: replace Abstract: The community is increasingly exploring linear RNNs (LRNNs) as language models, motivated by their expressive power and parallelizability.
By William Merrill, Hongjian Jiang, Yanhong Li, Anthony Lin, Ashish Sabharwal
arXiv:2605. 18079v2 Announce Type: replace Abstract: Existing expressivity results for transformers typically rely on hardmax attention, high precision, and other architectural modifications that disconnect them from the models used in practice.
By Moritz Br\"osamle, Stephan Eckstein
arXiv:2604. 25800v2 Announce Type: replace Abstract: Chain-of-Thought (CoT) has been shown to empirically improve Transformers' performance, and theoretically increase their expressivity to Turing completeness.
By Oliver Kraus, Yash Sarrof, Yuekun Yao, Alexander Koller, Michael Hahn
arXiv:2604. 14501v2 Announce Type: replace-cross Abstract: We study how depth, finite precision, state dimension, and chain-of-thought (CoT) affect the expressive power of multi-layer state-space models (SSMs).
By Nikola Zubi\'c, Qian Li, Yuyi Wang, Davide Scaramuzza
Modern large language models - transformers and diffusion language models - are built around two canonical algorithmic tasks: prediction and generation. We prove unconditional separations between low-depth quantum computation and the corresponding bounded-resource classical language-model architectures in both regimes.
arXiv:2609.15160v1 Announce Type: new
Abstract: Looped Transformers have recently demonstrated strong performance in both reasoning and language tasks by reusing a shared set of parameters across mul...
By Mingqian Yu, Wenpeng Zhang, Peilin Zhao