arXiv AI
Jun 18
Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers
arXiv:2503. 01805v3 Announce Type: replace-cross Abstract: Transformers have revolutionized the field of machine learning.
The Scaling Properties of Implicit Deductive Reasoning in Transformers
arXiv:2605. 04330v2 Announce Type: replace Abstract: We investigate the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers.
Related stories
arXiv AI
Aug 12
Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
arXiv:2604. 07822v2 Announce Type: replace-cross Abstract: We study implicit reasoning, i.
arXiv Machine Learning
Jul 23
Reproducing Recurrent Transformers: The CoTFormer
arXiv:2607. 19405v1 Announce Type: new Abstract: The CoTFormer architecture formalizes Chain-of-Thought as a form of recurrent latent computation, preserving intermediate states as attendable representations to mimic explicit reasoning traces.
arXiv AI
Jul 28
How Transformers Learn to Plan via Multi-Token Prediction
arXiv:2604. 11912v2 Announce Type: replace-cross Abstract: While next-token prediction (NTP) has been the standard objective for training language models, it often struggles to capture global structure in reasoning tasks.
arXiv AI
Jun 6
LoRi: Low-Rank Distillation for Implicit Reasoning
arXiv:2606. 05315v1 Announce Type: cross Abstract: Implicit chain-of-thought (iCoT) methods aim to internalize reasoning in large language models, but often underperform explicit CoT prompting.
arXiv AI
Aug 11
Think Deep, Speak Once: Relit, A Recursive Latent Implicit Transformer Framework
arXiv:2608. 08113v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has become the dominant paradigm for eliciting reasoning in Large Language Models (LLMs), yet it creates substantial computational overhead by forcing models to externalize intermediate reasoning steps as discrete tokens.
arXiv AI
Jun 8
A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning
arXiv:2606. 07410v1 Announce Type: cross Abstract: The emergence of "Aha moments" in large language models, particularly DeepSeek-R1-0120, has raised the question of whether these systems genuinely reason or merely imitate the appearance of reasoning.
arXiv AI
Aug 10
Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking
arXiv:2608. 07077v1 Announce Type: new Abstract: The Tower of Hanoi is a simple planning puzzle that in prior work has proven challenging for large reasoning models (LRMs).
arXiv AI
Jul 14
Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge
arXiv:2509. 24653v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) excel at multi-hop reasoning in distribution, yet fail on unseen compositions, a phenomenon known as the curse of two-hop reasoning.
arXiv Machine Learning
Jul 1
Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
arXiv:2606. 31779v1 Announce Type: new Abstract: Language models typically reason via explicit chain-of-thought (CoT), generating intermediate steps token-by-token.
arXiv AI
Jun 17
Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers
arXiv:2606. 18206v1 Announce Type: new Abstract: Looped architectures provide an inductive bias toward learning step-by-step procedures for tasks that require compositional reasoning.
arXiv AI
Jul 1
Deductive Logic in Language Models: Horizontal vs Vertical Reasoning
arXiv:2510. 09340v2 Announce Type: replace Abstract: Recent language models exhibit significant logical reasoning abilities, yet the mechanisms supporting deductive inference remain poorly understood.