arXiv:2606. 07720v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable reasoning abilities on mathematical and multi-hop planning tasks.
By Mujtaba Farhan, Maheep Chaudhary
arXiv:2608. 15703v1 Announce Type: new Abstract: Large language model (LLM) agents often perform poorly on complex, long-horizon tasks because their context becomes increasingly cluttered over time.
By XinQi Wang, Jinwei Xiao, Sijia Cui, Hongming Zhang, Yanna Wang, Qingyang Zhang, Bo Xu
arXiv:2607. 06648v1 Announce Type: new Abstract: Latent reasoning methods perform multi-step inference entirely in the model's continuous hidden states, promising more compact and efficient reasoning.
By Hengyu Jin, Shu Yang, Di Wang
The paper investigates Hierarchical Reasoning Models (HRM), a class of hierarchical Transformer-based latent reasoning models, across Sudoku, Maze, and ARC-AGI-2 tasks. By comparing HRM to Transformer baselines, applying causal interventions on recurrent states, and conducting linear probe and sparse autoencoder ablations, the authors find that recurrent models outperform one-pass baselines, that high- and low-level states contribute differently across tasks, and that ablations of sparse autoencoder features cause larger behavioral changes than probe-direction ablations. The study concludes that HRM implements constraint‑aware iterative refinement on a puzzle‑specific solution state, with component contributions varying without a compact, causally important feature set.
By Leo Raphael Rodrigues, Jian Kang
Latent Recurrent Thoughts (LRT) proposes a method for reasoning with frozen large language models by operating in the model’s continuous representation space. A small auxiliary network generates initial latent vectors, which a tiny recurrent reasoner refines over multiple steps, decoupling computational depth from model size. Experiments on symbolic and natural‑language reasoning tasks show that LRT outperforms prior frozen‑decoder continuous‑space methods and chain‑of‑thought prompting while using far less inference compute.
By Zhaoliang Chen, Jie Fu
The paper investigates whether different forms of intermediate computation in large language models—such as token-based traces, pause tokens, and latent reasoning—rely on the same underlying mechanism. By training five variants of GPTNeoX on an extended multi-hop reasoning task, the authors find that while vanilla, Chain-of-Thought, and Pause Token models perform well on in-distribution data, they fail to generalize to longer-hop out-of-distribution problems. In contrast, latent-reasoning models exhibit better depth generalization, with causal analysis revealing a sparse recurrent search circuit that implements forward reachability propagation across the graph.