arXiv AI
How Should Reasoning Be Organized in a Transformer's Latent Space?
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 15
How Many Thoughts Can a Vector Hold? The Capacity of Reasoning by Superposition
The paper investigates how continuous latent states in large language models can store multiple reasoning steps through superposition. It challenges the intuition that retaining only the current reasoning frontier is optimal, showing that cumulative superposition of the full reasoning history can actually require fewer representational dimensions. The authors demonstrate that this approach preserves more valid evidence, improves downstream outcome discrimination, and delays unreliability, while also establishing that uniform cumulative weighting of memories is minimax‑optimal for future reasoning.
arXiv Machine Learning
Jul 1
Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
arXiv:2606. 31779v1 Announce Type: new Abstract: Language models typically reason via explicit chain-of-thought (CoT), generating intermediate steps token-by-token.
arXiv AI
Jun 16
Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate
arXiv:2606. 16360v1 Announce Type: cross Abstract: Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead.
arXiv AI
Jul 17
T^2MLR: Transformer with Temporal Middle-Layer Recurrence
arXiv:2607. 15178v1 Announce Type: cross Abstract: Transformer reasoning is limited by autoregressive decoding, which repeat edly compresses rich hidden computation through token space and makes it difficult for intermediate reasoning states to persist across time.
arXiv AI
Jul 29
Penelope: Localized Latent Recurrence for Efficient Structured Reasoning
arXiv:2607. 25915v1 Announce Type: new Abstract: Complex structured reasoning tasks often require additional computation, yet current language models obtain it mainly by increasing parameter scale or by serializing intermediate steps as chain-of-thought (CoT) tokens.
arXiv AI
Jun 2
Latent Reasoning in TRMs is Secretly a Policy Improvement Operator
arXiv:2511. 16886v5 Announce Type: replace-cross Abstract: Recently, small models with latent recursion have obtained promising results on complex reasoning tasks.