The paper introduces Prototype-Mediated Process Supervision (PMPS), a method that uses learnable reasoning prototypes to provide structural supervision for latent chain-of-thought embeddings, addressing representation collapse and uneven information distribution. PMPS aligns latent and explicit CoT embeddings in a shared prototype space via many-to-many soft assignment and employs a Progressive Sequential Alignment module to guide training from positional priors to adaptive matching. Experiments show PMPS reduces output token length to under 50% of explicit CoT on GSM8K-Aug and improves accuracy by 2.08% over SIM-CoT, even surpassing CoT-SFT on GPT-2 and achieving the highest accuracy among latent reasoning methods on larger models and harder tasks.
arXiv:2607. 16972v1 Announce Type: new Abstract: Continuous Chain-of-Thought methods replace verbose reasoning traces with a short sequence of dense latent representations.
arXiv:2505.16782v3 Announce Type: replace Abstract: Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT...
Sep‑Latent introduces separator tokens as natural compression anchors for implicit reasoning in large language models, enabling the model to aggregate and compress reasoning logic into these anchors via an Attention‑Decay mechanism. The framework aligns explicit and implicit pathways through multi‑level distillation, ensuring logical consistency, and allows dynamic reasoning depth by terminating the latent thinking phase based on query complexity. On the GSM8k‑Aug benchmark, Sep‑Latent achieves 56.8% accuracy with an average of only 5.4 tokens, outperforming prior fixed‑length implicit models in efficiency while narrowing the gap with explicit Chain‑of‑Thought methods.
A*-Thought-V2 is a framework that models Chain-of-Thought reasoning as a geometric trajectory in a 3D PCA space, using explicit-implicit latent tokens to compress steps that deviate from the main question-to-solution direction. The method measures alignment angles to decide which steps remain text and which become latent, and introduces stepwise embedding forcing and label forcing to train the architecture. Experiments on Qwen models show up to 2.6% accuracy gains, halved response length, and significant reductions in computation and training time.
arXiv:2606. 06447v1 Announce Type: cross Abstract: Large language models often improve reasoning by generating explicit chain-of-thought (CoT), demonstrating the importance of intermediate computation.
The paper introduces LLoCoT, a looped latent‑reasoning framework that replaces the traditional left‑to‑right generation of chain‑of‑thought tokens with iterative refinement of a compact latent workspace using a shared transformer. By jointly updating latent slots over a few iterations and sampling latent tokens in parallel, LLoCoT feeds these into an autoregressive decoder for answer generation. Experiments on HumanEval and MBPP show that LLoCoT matches or surpasses explicit‑CoT baselines while dramatically reducing reasoning latency and increasing throughput.
arXiv:2608. 08113v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has become the dominant paradigm for eliciting reasoning in Large Language Models (LLMs), yet it creates substantial computational overhead by forcing models to externalize intermediate reasoning steps as discrete tokens.
arXiv:2606.13061v3 Announce Type: replace Abstract: Reasoning-driven universal multimodal embedding has advanced rapidly by introducing Chain-of-Thought (CoT) reasoning into the embedding pipeline. D...
Large language models often improve reasoning by generating explicit chain-of-thought (CoT), demonstrating the importance of intermediate computation. However, textual CoT forces this computation through a discrete, serial, and communication-oriented token stream: each reasoning step must be verbalized before the model can proceed, even when the underlying update is semantic, uncertain, or only partially formed.
arXiv:2606. 01243v1 Announce Type: cross Abstract: Latent reasoning enables Large Language Models (LLMs) to perform multi-step inference within continuous hidden states, offering efficiency gains over explicit Chain-of-Thought (CoT).
arXiv:2607. 25915v1 Announce Type: new Abstract: Complex structured reasoning tasks often require additional computation, yet current language models obtain it mainly by increasing parameter scale or by serializing intermediate steps as chain-of-thought (CoT) tokens.
The paper introduces LARK, a two‑stage latent reasoning framework designed to mitigate cross‑modal dilution in multimodal recommendation systems. In the first stage, learnable latent tokens are interleaved with chain‑of‑thought reasoning and aligned with a frozen vision encoder to preserve visual details. The second stage projects these latent representations through a bridge MLP, employing item‑to‑item contrastive learning and aligning intermediate features with the first‑stage hidden states to anchor final embeddings to the model’s reasoning output. Experiments on three public benchmarks and an industrial dataset demonstrate that LARK achieves state‑of‑the‑art performance across multiple recommendation architectures, with ablation studies confirming the contribution of each component.