arXiv Machine Learning By Franz Nowak, Ryan Cotterell, Reda Boumasmoud

An Algebraic View of the Expressivity of Recurrent Language Models

Read the original on arXiv Machine Learning →

arXiv:2606. 01765v1 Announce Type: cross Abstract: What formal languages can a recurrent neural language model recognize?

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI
Jun 9

MinMax Recurrent Neural Cascades

arXiv:2605. 06384v3 Announce Type: replace-cross Abstract: We introduce MinMax Recurrent Neural Cascades (MinMax RNCs), a class of recurrent neural networks built from a novel form of recurrence over the MinMax algebra.

By Alessandro Ronca
Hugging Face Trending Papers
Jun 16

An expressivity analysis of hierarchical modelling in deep transformers via bounded-depth grammars

Deep neural networks are widely believed to derive their expressive power from their ability to form \textbf{hierarchical representations}, capturing progressively more abstract and compositional features across layers. In language modeling, \textbf{transformers} have emerged as the dominant architecture, with early layers capturing local syntactic patterns and later layers encoding more complex clause-level dependencies.

arXiv Machine Learning
Jul 24

Compiling to recurrent neurons

arXiv:2511. 14953v2 Announce Type: replace-cross Abstract: Discrete structures are currently second-class in differentiable programming.

By Joey Velez-Ginorio, Nada Amin, Konrad Kording, Steve Zdancewic
arXiv Machine Learning
Jun 3

Why Are Linear RNNs More Parallelizable?

arXiv:2603. 03612v3 Announce Type: replace Abstract: The community is increasingly exploring linear RNNs (LRNNs) as language models, motivated by their expressive power and parallelizability.

By William Merrill, Hongjian Jiang, Yanhong Li, Anthony Lin, Ashish Sabharwal
arXiv Machine Learning
Jun 17

An expressivity analysis of hierarchical modelling in deep transformers via bounded-depth grammars

arXiv:2606. 17522v1 Announce Type: cross Abstract: Deep neural networks are widely believed to derive their expressive power from their ability to form \textbf{hierarchical representations}, capturing progressively more abstract and compositional features across layers.

By Vinoth Nandakumar, Qiang Qu, Pramod Thebe, Sakshi Khachariya, Tongliang Liu