arXiv AI By Varun Yerram, He He, Eunsol Choi

Training Continuous Chain of Thought Models: A Tale of Two Regimes

Read the original on arXiv AI →

arXiv:2607. 16972v1 Announce Type: new Abstract: Continuous Chain-of-Thought methods replace verbose reasoning traces with a short sequence of dense latent representations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 11

Structural Process Supervision for Latent Chain-of-Thought Reasoning

The paper introduces Prototype-Mediated Process Supervision (PMPS), a method that uses learnable reasoning prototypes to provide structural supervision for latent chain-of-thought embeddings, addressing representation collapse and uneven information distribution. PMPS aligns latent and explicit CoT embeddings in a shared prototype space via many-to-many soft assignment and employs a Progressive Sequential Alignment module to guide training from positional priors to adaptive matching. Experiments show PMPS reduces output token length to under 50% of explicit CoT on GSM8K-Aug and improves accuracy by 2.08% over SIM-CoT, even surpassing CoT-SFT on GPT-2 and achieving the highest accuracy among latent reasoning methods on larger models and harder tasks.

By Yiqi Li, Xu Chen, Chen Ju, Jiangchao Yao, Zhaoyang Li, Jinsong Lan, Xiaoyong Zhu, Bo Zheng, Yu Wang
arXiv Machine Learning
Jun 15

Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression

arXiv:2602. 08324v5 Announce Type: replace Abstract: Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs), yet it incurs substantial computational overhead for inference.

By Yuntian Tang, Bohan Jia, Wenxuan Huang, Lianyue Zhang, Jiao Xie, Wenxi Li, Wei Li, Jie Hu, Xinghao Chen Rongrong Ji, Shaohui Lin