arXiv Computation and Language September 10, 2026 By Sara Candussio, Daniel Scalena, Luca Bortolussi, Elisabetta Fersini, Malvina Nissim, Gabriele Sarti

Demystifying Entropy-based Selection for Chain-of-Thought Compression in Large Reasoning Models

Read the original on arXiv Computation and Language →

The paper evaluates entropy-based pruning for compressing Chain-of-Thought (CoT) reasoning in large models. Across multiple models and tasks, low- and high-entropy step selection shows no advantage over random pruning, and low-entropy token retention only helps on mathematical benchmarks due to the low entropy of numeric tokens. Patching a few CoT tokens with their original activations restores near-perfect performance, indicating that task information is distributed throughout the entire reasoning chain rather than concentrated in a small set of tokens.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

Sep 1

Every Token Leaves a Ripple in the Stream of Thought: Eliciting Model-Internal Token Saliency for Chain-of-Thought Compression

arXiv:2608.31066v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning improves multi-step problem solving, but long reasoning traces inflate inference cost. Token-level CoT compression red...

By Tianyi Zhao, Yinhan He, Wendy Zheng, Chen Chen

efficiency benchmarks

arXiv Machine Learning

Jun 15

Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression

arXiv:2602. 08324v5 Announce Type: replace Abstract: Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs), yet it incurs substantial computational overhead for inference.

By Yuntian Tang, Bohan Jia, Wenxuan Huang, Lianyue Zhang, Jiao Xie, Wenxi Li, Wei Li, Jie Hu, Xinghao Chen Rongrong Ji, Shaohui Lin

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 4

Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compression

arXiv:2510. 08647v2 Announce Type: replace-cross Abstract: Recent developments have enabled advanced reasoning in Large Language Models (LLMs) via long Chain-of-Thought (CoT), trading efficiency during inference for performance.

By Chengzhengxu Li, Xiaoming Liu, Zhaohan Zhang, Shengchao Liu, Guoxin Ma, Yu Lan, Cong Wang, Chao Shen

llms benchmarks

arXiv AI

Jun 15

Fractured Chain-of-Thought Reasoning

arXiv:2505. 12992v4 Announce Type: replace-cross Abstract: Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining.

By Baohao Liao, Hanze Dong, Yuhui Xu, Doyen Sahoo, Christof Monz, Junnan Li, Caiming Xiong

llms benchmarks

arXiv AI

Jul 14

Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought

arXiv:2607. 11266v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet it often incurs substantial computational costs due to over-reasoning: the generation of redundant, verbose, or irrelevant steps.

By Daeyeop Lee, Hwanjo Yu

llms efficiency benchmarks

arXiv AI

Jun 30

Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning

arXiv:2508. 02178v3 Announce Type: replace Abstract: Large reasoning models (LRMs) often exhibit overthinking, producing verbose Chain-of-Thought (CoT) traces that increase inference cost and obscure the underlying reasoning process.

By Taihang Zhen, Jialiang Hong, Kai Chen, Guang Yang, Junlan Feng, Wenpeng Zhu, Jing Huo, Yang Gao, Depeng Wang, Haitao Wan, Xi Yang, Fanyu Meng, Yuyao Zhang, Ji Qi, Xiangyu Zhou

reinforcement-learning

One email a morning, machine-written

Related stories

Every Token Leaves a Ripple in the Stream of Thought: Eliciting Model-Internal Token Saliency for Chain-of-Thought Compression

Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression

Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compression

Fractured Chain-of-Thought Reasoning

Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought

Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning