arXiv AI
Jul 8
Transformers converge to invariant algorithmic cores
arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.
Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does
Related stories
arXiv AI
Sep 16
Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization
arXiv:2603.21676v2 Announce Type: replace-cross Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....
arXiv AI
Jun 4
Outcome-Based RL Provably Leads Transformers to Reason, but Only With the Right Data
arXiv:2601. 15158v4 Announce Type: replace-cross Abstract: Transformers trained via Reinforcement Learning (RL) with outcome-based supervision can spontaneously develop the ability to generate intermediate reasoning steps (Chain-of-Thought).
arXiv AI
Jul 17
T^2MLR: Transformer with Temporal Middle-Layer Recurrence
arXiv:2607. 15178v1 Announce Type: cross Abstract: Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reasoning states to persist across time.
arXiv AI
Sep 3
Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?
The paper investigates whether a global workspace—a set of verbalisable, causally potent representations—emerges in transformer models that use recurrence instead of a stack of distinct layers. Using a Jacobian lens extended with a virtual‑unrolling adapter, the authors analyze two recurrent transformer architectures, Ouro‑2.6B and Huginn‑0125, and compare them to a standard Qwen3.6‑27B baseline. They find that a workspace does form in the iterated parts of both models, but recurrence alters how it can be accessed: Ouro reconstructs workspace content in every loop and requires writes and ablations across all loops, whereas Huginn forwards content across all recurrences but limits reads, writes, and ablations to a sliding window of about two recurrences.
arXiv Machine Learning
Jun 8
Predictive Statistics Shape Emergent World Representations of Grid Walkers
arXiv:2603. 16689v2 Announce Type: replace Abstract: Next-token predictors often appear to develop internal representations of the latent world and its rules.
arXiv AI
Sep 3
CHASE: Cache-Hole-Adapted Skip Exit for Looped State-Space Language Models
The paper introduces CHASE, a cache‑hole‑adapted skip‑exit mechanism for looped state‑space language models, specifically Looped Mamba and Looped Hybrid Mamba‑Transformer. It shows that looping these architectures improves performance on controlled reasoning tasks and remains competitive in pre‑training benchmarks while using fewer distinct parameters. The cache‑hole adaptation allows selective skipping of recurrent steps during inference, maintaining perplexity close to full computation and achieving significant speedups.
arXiv Machine Learning
Jul 2
Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training
arXiv:2607. 01232v1 Announce Type: new Abstract: Reinforcement learning (RL) has become a central component of post-training large language models (LLMs), yet little is understood about how RL adaptation is distributed across transformer layers.
arXiv Machine Learning
Jul 1
Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
arXiv:2606. 31779v1 Announce Type: new Abstract: Language models typically reason via explicit chain-of-thought (CoT), generating intermediate steps token-by-token.
arXiv AI
Jul 24
When Does Recurrence Become an Algorithm? Convergence Selection in Weight-Tied Looped Transformers
arXiv:2607. 20594v1 Announce Type: cross Abstract: When does a weight-tied looped transformer -- one block applied T times -- implement an actual algorithm?
arXiv AI
Jun 2
Latent Reasoning in TRMs is Secretly a Policy Improvement Operator
arXiv:2511. 16886v5 Announce Type: replace-cross Abstract: Recently, small models with latent recursion have obtained promising results on complex reasoning tasks.
arXiv AI
3d ago
How Should Reasoning Be Organized in a Transformer's Latent Space?
arXiv:2609.13747v2 Announce Type: replace Abstract: Continuous reasoning has emerged as a promising way to improve reasoning in large language models (LLMs). Yet we still lack a clear principle for d...