The paper investigates how continuous latent states in large language models can store multiple reasoning steps through superposition. It challenges the intuition that retaining only the current reasoning frontier is optimal, showing that cumulative superposition of the full reasoning history can actually require fewer representational dimensions. The authors demonstrate that this approach preserves more valid evidence, improves downstream outcome discrimination, and delays unreliability, while also establishing that uniform cumulative weighting of memories is minimax‑optimal for future reasoning.
By Hongyu Gu, Chang Liu, Jingwen Fu
Latent Recurrent Thoughts (LRT) proposes a method for reasoning with frozen large language models by operating in the model’s continuous representation space. A small auxiliary network generates initial latent vectors, which a tiny recurrent reasoner refines over multiple steps, decoupling computational depth from model size. Experiments on symbolic and natural‑language reasoning tasks show that LRT outperforms prior frozen‑decoder continuous‑space methods and chain‑of‑thought prompting while using far less inference compute.
By Zhaoliang Chen, Jie Fu
arXiv:2609.07406v1 Announce Type: new
Abstract: Chain-of-thought (CoT) reasoning improves the reasoning ability of large language models by introducing intermediate computation, but explicit rational...
By Yuwen Hao, Menglin Yang
The paper investigates the effectiveness of activation steering in latent chain-of-thought (CoT) reasoning compared to explicit CoT. It finds that steering continuous latent thoughts yields weaker impacts on language generation, even when hidden representations are shifted similarly. The authors propose a latent-to-language transition gap, supported by evidence of abrupt output distribution changes at the transition boundary and weaker bidirectional control in latent CoT.
By Gaoxiang Huang, Lei Qi
arXiv:2606. 16360v1 Announce Type: cross Abstract: Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead.
By Hanyu Lin, Min Cai, Jiawei Wen, Haodi Zhang
The paper investigates whether different forms of intermediate computation in large language models—such as token-based traces, pause tokens, and latent reasoning—rely on the same underlying mechanism. By training five variants of GPTNeoX on an extended multi-hop reasoning task, the authors find that while vanilla, Chain-of-Thought, and Pause Token models perform well on in-distribution data, they fail to generalize to longer-hop out-of-distribution problems. In contrast, latent-reasoning models exhibit better depth generalization, with causal analysis revealing a sparse recurrent search circuit that implements forward reachability propagation across the graph.
arXiv:2609.23065v1 Announce Type: cross
Abstract: Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through inp...
By Qianli Wang, Yilong Wang, Dennis Wei, Jingyi Sun, Simon Ostermann, Pepa Atanasova, Nils Feldhus
A*-Thought-V2 is a framework that models Chain-of-Thought reasoning as a geometric trajectory in a 3D PCA space, using explicit-implicit latent tokens to compress steps that deviate from the main question-to-solution direction. The method measures alignment angles to decide which steps remain text and which become latent, and introduces stepwise embedding forcing and label forcing to train the architecture. Experiments on Qwen models show up to 2.6% accuracy gains, halved response length, and significant reductions in computation and training time.
By Xiaoang Xu, Siyuan Liu, Shuo Wang, Junlan Feng, Fanyu Meng, Zhu Zhang, Jixun Wang, Xiaorong Wang, Zihan Zhou, Xin Li, Chaojun Xiao, Yiming Zhang, Huijia Wu, Liuyu Xiang, Peipei Li, Zhaofeng He
arXiv:2603. 01437v2 Announce Type: replace Abstract: As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning.
By Kyle Cox, Darius Kianersi, Adri\`a Garriga-Alonso
arXiv:2607. 21433v1 Announce Type: cross Abstract: Chain-of-thought reasoning models such as DeepSeek-R1-Distill-Qwen-7B exhibit a bimodal convergence pattern: generations either terminate within a token budget (converged) or exhaust it without reaching a conclusion (non-converged).
By Renuka Oladri, Niveda Jawahar, Abdirisak Mohamed
arXiv:2505.16782v3 Announce Type: replace
Abstract: Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT...
By Xinghao Chen, Anhao Zhao, Heming Xia, Xuan Lu, Hanlin Wang, Yanjun Chen, Wei Zhang, Jian Wang, Wenjie Li, Xiaoyu Shen
arXiv:2412.06769v4 Announce Type: replace
Abstract: Large language models (LLMs) are typically constrained to reason in the language space, where they express the reasoning process through a chain-of...
By Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, Yuandong Tian