arXiv:2608. 15703v1 Announce Type: new Abstract: Large language model (LLM) agents often perform poorly on complex, long-horizon tasks because their context becomes increasingly cluttered over time.
Large language model (LLM) agents often perform poorly on complex, long-horizon tasks because their context becomes increasingly cluttered over time. As interactions accumulate, detailed execution tra...
arXiv:2607. 25915v1 Announce Type: new Abstract: Complex structured reasoning tasks often require additional computation, yet current language models obtain it mainly by increasing parameter scale or by serializing intermediate steps as chain-of-thought (CoT) tokens.
The paper investigates Hierarchical Reasoning Models (HRM), a class of hierarchical Transformer-based latent reasoning models, across Sudoku, Maze, and ARC-AGI-2 tasks. By comparing HRM to Transformer baselines, applying causal interventions on recurrent states, and conducting linear probe and sparse autoencoder ablations, the authors find that recurrent models outperform one-pass baselines, that high- and low-level states contribute differently across tasks, and that ablations of sparse autoencoder features cause larger behavioral changes than probe-direction ablations. The study concludes that HRM implements constraint‑aware iterative refinement on a puzzle‑specific solution state, with component contributions varying without a compact, causally important feature set.
arXiv:2609.39967v1 Announce Type: cross Abstract: Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few...
arXiv:2608. 08113v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has become the dominant paradigm for eliciting reasoning in Large Language Models (LLMs), yet it creates substantial computational overhead by forcing models to externalize intermediate reasoning steps as discrete tokens.
The paper investigates whether different forms of intermediate computation in large language models—such as token-based traces, pause tokens, and latent reasoning—rely on the same underlying mechanism. By training five variants of GPTNeoX on an extended multi-hop reasoning task, the authors find that while vanilla, Chain-of-Thought, and Pause Token models perform well on in-distribution data, they fail to generalize to longer-hop out-of-distribution problems. In contrast, latent-reasoning models exhibit better depth generalization, with causal analysis revealing a sparse recurrent search circuit that implements forward reachability propagation across the graph.
arXiv:2603.21676v2 Announce Type: replace-cross Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....
Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them strong on algorithmic ta...
Latent Recurrent Thoughts (LRT) proposes a method for reasoning with frozen large language models by operating in the model’s continuous representation space. A small auxiliary network generates initial latent vectors, which a tiny recurrent reasoner refines over multiple steps, decoupling computational depth from model size. Experiments on symbolic and natural‑language reasoning tasks show that LRT outperforms prior frozen‑decoder continuous‑space methods and chain‑of‑thought prompting while using far less inference compute.
arXiv:2609.33149v2 Announce Type: replace Abstract: A common principle of effective learning is to practice material that is neither already mastered nor too difficult to permit progress. We ask how...
arXiv:2603. 02112v2 Announce Type: replace Abstract: Modern language models reason within bounded context, an inherent constraint that poses a fundamental barrier to long-horizon reasoning.