arXiv AI
Jun 2
BRo-JEPA: Learning Modular Arithmetic in Latent Space
arXiv:2606. 01372v1 Announce Type: cross Abstract: Can neural networks learn abstract algebraic rules, or do they merely memorize training patterns?
Machine learning modularity
The paper introduces a transformer‑based sequence‑to‑sequence model with dynamic batching that automatically simplifies complex expressions involving multiple elliptic Gamma functions. It learns to apply algebraic identities, especially SL(2,ℤ) and SL(3,ℤ) modular transformations, to reduce scrambled expressions to canonical forms. Experiments show over 99% accuracy on in‑distribution tests and more than 90% accuracy under significant extrapolation, indicating the model internalizes the underlying algebraic rules rather than merely memorizing patterns.
Related stories
arXiv AI
Aug 19
BRo-JEPA: Learning Modular Transformations in Latent Space
The paper introduces BRo-JEPA, a world model that learns modular arithmetic operations as rotations in latent space. Using MNIST and EMNIST datasets, BRo-JEPA achieves near-perfect zero‑shot generalization to unseen operations, outperforming standard supervised and JEPA baselines by a large margin. The model demonstrates that latent transformations can encode underlying algebraic structures, enabling strict zero‑shot operation generalization.
arXiv Machine Learning
Jun 15
IntSeqBERT: Learning Arithmetic Structure in OEIS via Modulo-Spectrum Embeddings
arXiv:2603. 05556v2 Announce Type: replace Abstract: Integer sequences in the OEIS span values from single-digit constants to astronomical factorials and exponentials, making prediction challenging for standard tokenised models that cannot handle out-of-vocabulary values or exploit periodic arithmetic structure.
arXiv AI
Jul 9
Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits
arXiv:2607. 07066v1 Announce Type: cross Abstract: Transformers have demonstrated a remarkable ability to learn algorithmic reasoning, yet mechanistic analyses have mostly focused on globally invertible operations such as cyclic addition and group composition.
arXiv Machine Learning
Aug 13
A Theoretical Framework for Modular Learning of Robust Generative Models
arXiv:2602. 17554v3 Announce Type: replace Abstract: Training large-scale generative models is resource-intensive and relies heavily on heuristic dataset weighting.
arXiv Machine Learning
Jun 8
Discovering Multiscale Deep Formulas in Complex Systems via Neural-Guided Lambda Calculus
arXiv:2606. 07426v1 Announce Type: new Abstract: A fundamental problem in science is identifying underlying patterns of complex systems in the form of concise mathematical formulas.
arXiv AI
Oct 9
Probability-Signature Dynamics: Unpacking Modular Addition Learning Within Two-Layer Networks
The paper investigates why neural networks trained on modular addition tasks develop Fourier-structured representations that enable exact generalization. Using probability signatures, the authors show that leading gradient interactions are cyclic shift operators, which are diagonalized by the discrete Fourier transform, leading to decoupled Fourier-mode dynamics that explain Fourier sparsity, frequency matching, and phase alignment. The framework also clarifies how label noise can accelerate early loss decrease by increasing conditional label collisions, and it is demonstrated to predict frequency behavior in other operators such as XOR.
arXiv Machine Learning
Jul 21
Explaining and Tuning Transformer-based LLMs in Arithmetic Tasks with Human Strategies
arXiv:2607. 17166v1 Announce Type: new Abstract: Transformer-based large language models (LLMs) continue to achieve state-of-the-art performance across various natural language processing tasks.
arXiv Machine Learning
Oct 12
Algebraic Integer Quantization for Machine Learning Applications
The paper introduces algebraic integer number formats (AINF) as a framework for highly configurable and accurate compressed computation, emphasizing its reduction to purely integer arithmetic for error‑free multiply‑accumulate operations. A matrix multiplication engine leveraging AINF achieves performance close to the most efficient numerical counterparts on modern hardware accelerators. The authors further propose algebraic integer quantization (AIQ) as a lossy compression format for large language model inference, demonstrating competitive accuracy against the industry‑standard MX format at equivalent bit‑per‑parameter budgets, and suggest that higher‑order algebraic integers could yield even greater computational benefits.
arXiv AI
Jun 10
ASyMOB: Algebraic Symbolic Mathematical Operations Benchmark
arXiv:2505. 23851v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly applied to symbolic mathematics, yet existing evaluations often conflate pattern memorization with genuine reasoning.
arXiv Machine Learning
Jul 9
FPTQuant: Function-Preserving Transforms for LLM Quantization
arXiv:2506. 04985v2 Announce Type: replace Abstract: Large language models (LLMs) require substantial compute, and thus energy, at inference time.
arXiv AI
Jul 15
Prime Fourier Embeddings: A Principled Basis for Modular Arithmetic
arXiv:2606. 23044v2 Announce Type: replace-cross Abstract: Numbers have algebraic structure that standard neural embeddings often fail to expose.