arXiv AI
22h ago
BRo-JEPA: Learning Modular Transformations in Latent Space
arXiv:2606. 01372v2 Announce Type: replace-cross Abstract: Can neural networks learn algebraic rules from visual inputs, or do they merely fit observed patterns?
BRo-JEPA: Learning Modular Arithmetic in Latent Space
arXiv:2606. 01372v1 Announce Type: cross Abstract: Can neural networks learn abstract algebraic rules, or do they merely memorize training patterns?
Related stories
arXiv AI
Jul 9
Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits
arXiv:2607. 07066v1 Announce Type: cross Abstract: Transformers have demonstrated a remarkable ability to learn algorithmic reasoning, yet mechanistic analyses have mostly focused on globally invertible operations such as cyclic addition and group composition.
arXiv AI
Jun 15
Numbers Already Carry Their Own Embeddings
arXiv:2606. 14108v1 Announce Type: cross Abstract: We introduce Adelic operation-preserved embeddings (AOE), a training-free representation that captures both a number's real value and its modular (p-adic) signatures.
arXiv Machine Learning
Jun 11
Composing Linear Layers from Irreducibles
arXiv:2507. 11688v4 Announce Type: replace Abstract: Contemporary large models often exhibit behaviors suggesting the presence of low-level primitives that compose into modules with richer functionality, but these fundamental building blocks remain poorly understood.
arXiv Machine Learning
Jun 16
Next-Latent Prediction Transformers Learn Compact World Models
arXiv:2511. 05963v4 Announce Type: replace Abstract: Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens.
arXiv Machine Learning
Jun 8
Skip a Layer or Loop It? Learning Program-of-Layers in LLMs
arXiv:2606. 06574v1 Announce Type: new Abstract: Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers.
arXiv AI
Jun 29
The Context-Ready Transformer
arXiv:2606. 27538v1 Announce Type: cross Abstract: We introduce the context-ready transformer, a new recurrent neural network architecture built from a D-layer transformer block that pre-contextualizes each token before it enters the block.
arXiv AI
22h ago
Towards Zero-Shot Task Transfer with Neurosymbolic World Models
arXiv:2608. 17959v1 Announce Type: new Abstract: State-of-the-art model-based reinforcement learning methods learn neural world models that allow policy improvement by planning in a latent space, without assumptions on the structure of the underlying environment.
arXiv Machine Learning
Jul 16
Targeted Recovery of Weight-Space Mechanisms From Neural Networks
arXiv:2607. 13047v1 Announce Type: new Abstract: Parameter decomposition (PD) decomposes neural networks into interpretable computational components that faithfully reflect the original network's operations.
arXiv Machine Learning
Jul 1
Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
arXiv:2606. 31779v1 Announce Type: new Abstract: Language models typically reason via explicit chain-of-thought (CoT), generating intermediate steps token-by-token.
arXiv Machine Learning
Jun 25
RotRNN: Modelling Long Sequences with Rotations
arXiv:2407. 07239v3 Announce Type: replace Abstract: Linear recurrent neural networks, such as State Space Models (SSMs) and Linear Recurrent Units (LRUs), have recently shown state-of-the-art performance on long sequence modelling benchmarks.
arXiv AI
Aug 5
State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking
arXiv:2608. 03425v1 Announce Type: new Abstract: Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms.