arXiv AI September 18, 2026 By Khashayar Gatmiry, Avrajit Ghosh, Parsa Mirtaheri, Jason D. Lee, Nika Haghtalab, Emmanuel Abbe, Peter Bartlett

Learn Your Own Thoughts: Abstract Token Curriculum

Read the original on arXiv AI →

The paper introduces Abstract Token Curriculum (ATC), a curriculum learning framework that enables large language models to develop continuous intermediate representations—referred to as abstract thoughts—without explicit supervision or manual scratchpad design. ATC incrementally raises problem difficulty through a sequence of distributions, guiding models to focus attention on the most informative tokens for predicting subsequent tokens. The authors provide theoretical analysis for parity function learning with single‑layer softmax attention and demonstrate ATC’s effectiveness on graph reachability and arithmetic learning tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

llms

arXiv AI

Jul 21

Training Continuous Chain of Thought Models: A Tale of Two Regimes

arXiv:2607. 16972v1 Announce Type: new Abstract: Continuous Chain-of-Thought methods replace verbose reasoning traces with a short sequence of dense latent representations.

By Varun Yerram, He He, Eunsol Choi

rag

arXiv Computation and Language

Aug 25

Training Large Language Models to Reason in a Continuous Latent Space

arXiv:2412.06769v4 Announce Type: replace Abstract: Large language models (LLMs) are typically constrained to reason in the language space, where they express the reasoning process through a chain-of...

By Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, Yuandong Tian

llms rag diffusion

arXiv AI

Jun 6

CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning

arXiv:2509. 04027v3 Announce Type: replace Abstract: Test-time scaling, primarily manifested through multi-step Chain-of-Thought (CoT) reasoning via Reinforcement Learning (RL), has emerged as a pivotal paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs).

By Zeyu Gan, Hao Yi, Yong Liu

llms reinforcement-learning

arXiv AI

Sep 1

Learning Composable Chains-of-Thought

arXiv:2505.22635v2 Announce Type: replace-cross Abstract: A common approach for teaching large language models (LLMs) to reason is to train on chain-of-thought (CoT) traces of in-distribution reasoni...

By Fangcong Yin, Zeyu Leo Liu, Liu Leqi, Xi Ye, Greg Durrett

llms fine-tuning

arXiv AI

Aug 11

ZetaGPT: A Reference Implementation of Positional--Encoding--Free State--Space--Attention Language Models

arXiv:2608. 09432v1 Announce Type: cross Abstract: Transformer-based language models rely on self-attention, whose computation is permutation-equivariant and therefore lacks an intrinsic mechanism for representing token order.

By R\'ois\'in Luo

llms rag nlp reinforcement-learning fine-tuning

arXiv AI

Jun 15

SuperThoughts: Reasoning Tokens in Superposition

arXiv:2606. 13862v1 Announce Type: cross Abstract: Long Chain-of-Thought (CoT) reasoning improves LLM problem-solving but is computationally expensive due to sequential token generation.

By Zheyang Xiong, Shivam Garg, Max Yu, Vaishnavi Shrivastava, Haoyu Zhao, Anastasios Kyrillidis, Dimitris Papailiopoulos

llms efficiency

One email a morning, machine-written

Related stories

Training Continuous Chain of Thought Models: A Tale of Two Regimes

Training Large Language Models to Reason in a Continuous Latent Space

CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning

Learning Composable Chains-of-Thought

ZetaGPT: A Reference Implementation of Positional--Encoding--Free State--Space--Attention Language Models

SuperThoughts: Reasoning Tokens in Superposition