A*-Thought-V2 is a framework that models Chain-of-Thought reasoning as a geometric trajectory in a 3D PCA space, using explicit-implicit latent tokens to compress steps that deviate from the main question-to-solution direction. The method measures alignment angles to decide which steps remain text and which become latent, and introduces stepwise embedding forcing and label forcing to train the architecture. Experiments on Qwen models show up to 2.6% accuracy gains, halved response length, and significant reductions in computation and training time.
arXiv:2606. 16360v1 Announce Type: cross Abstract: Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead.
arXiv:2608. 13570v1 Announce Type: cross Abstract: Latent reasoning has emerged as a powerful alternative to text-based Chain-of-Thought (CoT), offering significant gains in computational efficiency by compressing verbose reasoning into compact embeddings.
Latent Recurrent Thoughts (LRT) proposes a method for reasoning with frozen large language models by operating in the model’s continuous representation space. A small auxiliary network generates initial latent vectors, which a tiny recurrent reasoner refines over multiple steps, decoupling computational depth from model size. Experiments on symbolic and natural‑language reasoning tasks show that LRT outperforms prior frozen‑decoder continuous‑space methods and chain‑of‑thought prompting while using far less inference compute.
Sep‑Latent introduces separator tokens as natural compression anchors for implicit reasoning in large language models, enabling the model to aggregate and compress reasoning logic into these anchors via an Attention‑Decay mechanism. The framework aligns explicit and implicit pathways through multi‑level distillation, ensuring logical consistency, and allows dynamic reasoning depth by terminating the latent thinking phase based on query complexity. On the GSM8k‑Aug benchmark, Sep‑Latent achieves 56.8% accuracy with an average of only 5.4 tokens, outperforming prior fixed‑length implicit models in efficiency while narrowing the gap with explicit Chain‑of‑Thought methods.
INTACT introduces a small external encoder that induces latent reasoning in a frozen backbone model by mapping hidden states to compact continuous representations, called soft-token blocks. The encoder is trained via distillation to mimic written reasoning steps while preserving the backbone’s behavior, and then fine‑tuned for answer correctness. Experiments on four frozen backbones (1B–8B) show that INTACT improves accuracy on GSM8K and other out‑of‑domain benchmarks, reduces latency by up to 19%, and outperforms existing latent‑reasoning baselines at 3B and 8B scales.
arXiv:2505.16782v3 Announce Type: replace Abstract: Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT...
arXiv:2609.07406v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning improves the reasoning ability of large language models by introducing intermediate computation, but explicit rational...
The paper introduces LLoCoT, a looped latent‑reasoning framework that replaces the traditional left‑to‑right generation of chain‑of‑thought tokens with iterative refinement of a compact latent workspace using a shared transformer. By jointly updating latent slots over a few iterations and sampling latent tokens in parallel, LLoCoT feeds these into an autoregressive decoder for answer generation. Experiments on HumanEval and MBPP show that LLoCoT matches or surpasses explicit‑CoT baselines while dramatically reducing reasoning latency and increasing throughput.
arXiv:2606. 06447v1 Announce Type: cross Abstract: Large language models often improve reasoning by generating explicit chain-of-thought (CoT), demonstrating the importance of intermediate computation.
The paper introduces Prototype-Mediated Process Supervision (PMPS), a method that uses learnable reasoning prototypes to provide structural supervision for latent chain-of-thought embeddings, addressing representation collapse and uneven information distribution. PMPS aligns latent and explicit CoT embeddings in a shared prototype space via many-to-many soft assignment and employs a Progressive Sequential Alignment module to guide training from positional priors to adaptive matching. Experiments show PMPS reduces output token length to under 50% of explicit CoT on GSM8K-Aug and improves accuracy by 2.08% over SIM-CoT, even surpassing CoT-SFT on GPT-2 and achieving the highest accuracy among latent reasoning methods on larger models and harder tasks.
Latent reasoning replaces natural-language chains of thought with compact continuous "soft-token" representations, reducing the number of intermediate tokens that must be decoded autoregressively. We...
The paper investigates whether different forms of intermediate computation in large language models—such as token-based traces, pause tokens, and latent reasoning—rely on the same underlying mechanism. By training five variants of GPTNeoX on an extended multi-hop reasoning task, the authors find that while vanilla, Chain-of-Thought, and Pause Token models perform well on in-distribution data, they fail to generalize to longer-hop out-of-distribution problems. In contrast, latent-reasoning models exhibit better depth generalization, with causal analysis revealing a sparse recurrent search circuit that implements forward reachability propagation across the graph.