arXiv Machine Learning
Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 8
Transformers converge to invariant algorithmic cores
arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.
arXiv AI
Sep 16
Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization
arXiv:2603.21676v2 Announce Type: replace-cross Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....
arXiv AI
Jun 4
Outcome-Based RL Provably Leads Transformers to Reason, but Only With the Right Data
arXiv:2601. 15158v4 Announce Type: replace-cross Abstract: Transformers trained via Reinforcement Learning (RL) with outcome-based supervision can spontaneously develop the ability to generate intermediate reasoning steps (Chain-of-Thought).
arXiv AI
Jul 17
T^2MLR: Transformer with Temporal Middle-Layer Recurrence
arXiv:2607. 15178v1 Announce Type: cross Abstract: Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reasoning states to persist across time.
arXiv AI
Sep 3
Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?
The paper investigates whether a global workspace—a set of verbalisable, causally potent representations—emerges in transformer models that use recurrence instead of a stack of distinct layers. Using a Jacobian lens extended with a virtual‑unrolling adapter, the authors analyze two recurrent transformer architectures, Ouro‑2.6B and Huginn‑0125, and compare them to a standard Qwen3.6‑27B baseline. They find that a workspace does form in the iterated parts of both models, but recurrence alters how it can be accessed: Ouro reconstructs workspace content in every loop and requires writes and ablations across all loops, whereas Huginn forwards content across all recurrences but limits reads, writes, and ablations to a sliding window of about two recurrences.
arXiv Machine Learning
Jun 8
Predictive Statistics Shape Emergent World Representations of Grid Walkers
arXiv:2603. 16689v2 Announce Type: replace Abstract: Next-token predictors often appear to develop internal representations of the latent world and its rules.