arXiv AI
Jun 18
Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers
arXiv:2503. 01805v3 Announce Type: replace-cross Abstract: Transformers have revolutionized the field of machine learning.
The Scaling Properties of Implicit Deductive Reasoning in Transformers
arXiv:2605. 04330v2 Announce Type: replace Abstract: We investigate the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers.
Related stories
arXiv AI
Aug 12
Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
arXiv:2604. 07822v2 Announce Type: replace-cross Abstract: We study implicit reasoning, i.
arXiv Machine Learning
Jul 23
Reproducing Recurrent Transformers: The CoTFormer
arXiv:2607. 19405v1 Announce Type: new Abstract: The CoTFormer architecture formalizes Chain-of-Thought as a form of recurrent latent computation, preserving intermediate states as attendable representations to mimic explicit reasoning traces.
arXiv AI
Sep 16
Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization
arXiv:2603.21676v2 Announce Type: replace-cross Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....
arXiv AI
Jul 28
How Transformers Learn to Plan via Multi-Token Prediction
arXiv:2604. 11912v2 Announce Type: replace-cross Abstract: While next-token prediction (NTP) has been the standard objective for training language models, it often struggles to capture global structure in reasoning tasks.
arXiv AI
Jun 6
LoRi: Low-Rank Distillation for Implicit Reasoning
arXiv:2606. 05315v1 Announce Type: cross Abstract: Implicit chain-of-thought (iCoT) methods aim to internalize reasoning in large language models, but often underperform explicit CoT prompting.
arXiv AI
Sep 15
Tool Use Reduces Depth-Induced Collapse in OOD Reasoning
arXiv:2602.21061v2 Announce Type: replace Abstract: Many current paths to more advanced AI depend on the assumption that large language models (LLMs) can generalize learned relationships to solve com...
arXiv AI
Aug 11
Think Deep, Speak Once: Relit, A Recursive Latent Implicit Transformer Framework
arXiv:2608. 08113v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has become the dominant paradigm for eliciting reasoning in Large Language Models (LLMs), yet it creates substantial computational overhead by forcing models to externalize intermediate reasoning steps as discrete tokens.
arXiv AI
Jun 8
A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning
arXiv:2606. 07410v1 Announce Type: cross Abstract: The emergence of "Aha moments" in large language models, particularly DeepSeek-R1-0120, has raised the question of whether these systems genuinely reason or merely imitate the appearance of reasoning.
arXiv AI
Aug 10
Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking
arXiv:2608. 07077v1 Announce Type: new Abstract: The Tower of Hanoi is a simple planning puzzle that in prior work has proven challenging for large reasoning models (LRMs).
arXiv AI
Jul 14
Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge
arXiv:2509. 24653v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) excel at multi-hop reasoning in distribution, yet fail on unseen compositions, a phenomenon known as the curse of two-hop reasoning.
arXiv Machine Learning
Sep 1
Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers
arXiv:2608.31067v1 Announce Type: new Abstract: Learning generalizable algorithmic computations remains a challenge for neural networks, as reflected in persistent failures on compositional and lengt...