arXiv:2606. 23044v2 Announce Type: replace-cross Abstract: Numbers have algebraic structure that standard neural embeddings often fail to expose.
arXiv:2603. 05556v2 Announce Type: replace Abstract: Integer sequences in the OEIS span values from single-digit constants to astronomical factorials and exponentials, making prediction challenging for standard tokenised models that cannot handle out-of-vocabulary values or exploit periodic arithmetic structure.
arXiv:2607. 07066v1 Announce Type: cross Abstract: Transformers have demonstrated a remarkable ability to learn algorithmic reasoning, yet mechanistic analyses have mostly focused on globally invertible operations such as cyclic addition and group composition.
arXiv:2606. 14108v1 Announce Type: cross Abstract: We introduce Adelic operation-preserved embeddings (AOE), a training-free representation that captures both a number's real value and its modular (p-adic) signatures.
arXiv:2606. 01372v1 Announce Type: cross Abstract: Can neural networks learn abstract algebraic rules, or do they merely memorize training patterns?
The paper introduces BRo-JEPA, a world model that learns modular arithmetic operations as rotations in latent space. Using MNIST and EMNIST datasets, BRo-JEPA achieves near-perfect zero‑shot generalization to unseen operations, outperforming standard supervised and JEPA baselines by a large margin. The model demonstrates that latent transformations can encode underlying algebraic structures, enabling strict zero‑shot operation generalization.
arXiv:2507. 11688v4 Announce Type: replace Abstract: Contemporary large models often exhibit behaviors suggesting the presence of low-level primitives that compose into modules with richer functionality, but these fundamental building blocks remain poorly understood.
arXiv:2606. 17399v1 Announce Type: cross Abstract: When small transformers grok modular multiplication, prior work reports that the learned embedding has a "dense" Fourier spectrum requiring all frequencies.
arXiv:2606. 08985v1 Announce Type: new Abstract: While neural collapse (NC) predicts that a $K$-class-balanced classifier should organize terminal representations as a $(K-1)$-dimensional simplex equiangular tight frame (ETF), modular addition consistently enters a different regime: networks compress to a two-dimensional cyclic geometry in which both classifier weights and token embeddings lie on circles.
arXiv:2601. 11618v3 Announce Type: replace-cross Abstract: Neural architecture is often identified by module syntax, computation graphs, or the composite functions they realize.
arXiv:2609.28409v1 Announce Type: cross Abstract: Vector Symbolic Algebras project data structures into a hyperdimensional vector space through the application of their vector algebras to randomly ge...
arXiv:2606. 02993v1 Announce Type: new Abstract: Understanding how structured internal structure emerges during neural network training is central to the study of deep learning.
arXiv:2607. 13749v1 Announce Type: new Abstract: Neural networks trained on modular arithmetic exhibit grokking, a delayed transition from memorisation to generalisation known to depend on model capacity: too little and the network memorises slowly or not at all, too much and it generalises almost immediately.