arXiv Machine Learning
Jun 19
Recurrent neural networks approximate continuous functions
arXiv:2606. 20325v1 Announce Type: new Abstract: Classical approximation theorems ask for a new neural network whenever the target accuracy is improved.
An Algebraic View of the Expressivity of Recurrent Language Models
arXiv:2606. 01765v1 Announce Type: cross Abstract: What formal languages can a recurrent neural language model recognize?
Related stories
arXiv AI
Jun 9
MinMax Recurrent Neural Cascades
arXiv:2605. 06384v3 Announce Type: replace-cross Abstract: We introduce MinMax Recurrent Neural Cascades (MinMax RNCs), a class of recurrent neural networks built from a novel form of recurrence over the MinMax algebra.
Hugging Face Trending Papers
Jun 16
An expressivity analysis of hierarchical modelling in deep transformers via bounded-depth grammars
Deep neural networks are widely believed to derive their expressive power from their ability to form \textbf{hierarchical representations}, capturing progressively more abstract and compositional features across layers. In language modeling, \textbf{transformers} have emerged as the dominant architecture, with early layers capturing local syntactic patterns and later layers encoding more complex clause-level dependencies.
arXiv Machine Learning
Jul 24
Compiling to recurrent neurons
arXiv:2511. 14953v2 Announce Type: replace-cross Abstract: Discrete structures are currently second-class in differentiable programming.
arXiv Machine Learning
Jun 3
Why Are Linear RNNs More Parallelizable?
arXiv:2603. 03612v3 Announce Type: replace Abstract: The community is increasingly exploring linear RNNs (LRNNs) as language models, motivated by their expressive power and parallelizability.
arXiv Machine Learning
Jun 17
An expressivity analysis of hierarchical modelling in deep transformers via bounded-depth grammars
arXiv:2606. 17522v1 Announce Type: cross Abstract: Deep neural networks are widely believed to derive their expressive power from their ability to form \textbf{hierarchical representations}, capturing progressively more abstract and compositional features across layers.
arXiv Machine Learning
Jun 11
Why Depth Matters in Parallelizable Sequence Models: A Lie Algebraic View
arXiv:2603. 05573v2 Announce Type: replace Abstract: Scalable sequence models, such as Transformer variants and structured state-space models, often trade expressivity power for sequence-level parallelism, which enables efficient training.
arXiv AI
Jun 3
The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models
arXiv:2606. 03645v1 Announce Type: cross Abstract: Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output.
arXiv AI
Jun 2
BRo-JEPA: Learning Modular Arithmetic in Latent Space
arXiv:2606. 01372v1 Announce Type: cross Abstract: Can neural networks learn abstract algebraic rules, or do they merely memorize training patterns?
arXiv Machine Learning
Jul 30
A Compositional Theory of Causally Masked Transformers
arXiv:2607. 26988v1 Announce Type: cross Abstract: What types of decision problems can a causally masked, finite-precision transformer solve for inputs of arbitrary length?
arXiv Machine Learning
Jul 27
Relaxed activation analysis of dataflow networks - A clock calculus for machine learning and real-time scheduling
arXiv:2607. 21797v1 Announce Type: cross Abstract: Previous work has shown that the simple dataflow primitives of the Lustre language allow the natural, semantically unambiguous, and compact representation of machine learning (ML) applications, including models featuring complex conditional execution and recurrent state.
arXiv Machine Learning
Jun 4
On the Expressive Power of Permutation-Equivariant Weight-Space Networks
arXiv:2602. 01083v2 Announce Type: replace Abstract: Weight-space learning studies neural architectures that operate directly on the parameters of other neural networks.