← Back to all news
arXiv AI By Tanush Kalhan, Anupam Chattopadhyay, Sriram Ranga, Amritendu Mukherjee

The Density of Thought: Predicting and Steering Reasoning in Large Language Models

Read the original on arXiv AI →

The paper studies how the distribution of information across chain‑of‑thought steps—measured by LP variance and Caveman compression ratio—relates to answer correctness in large language models. Across six model sizes, two training paradigms, and 60 GSM8K problems, it finds that lower LP variance predicts accuracy in supervised fine‑tuned models but not in reinforcement‑learning distilled ones, that models prefer more complex distributions, and that steering density at inference time harms accuracy while pre‑generation nudging and a new Entropy‑Gated Density Routing method improve it. The work demonstrates that natural density patterns are functionally important for reasoning and offers practical techniques to enhance performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 1

Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training

The paper investigates how different forms of compressed chain‑of‑thought (CoT) reasoning—Explicit, Composed, and Implicit—affect large language model (LLM) performance after supervised fine‑tuning (SFT). Using a synthetic compositional reasoning task, the authors show that coarser CoT requires more SFT data, that Composed and Implicit CoT benefit more from data scaling (with Composed also benefiting from repetition), and that reinforcement learning with verifiable rewards (RLVR) can decompose compressed steps learned during SFT. Additionally, unidirectional CoT ordering improves generalization on longer sequential tasks.

By Kohsei Matsutani, Gouki Minegishi, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo
llmsreinforcement-learningfine-tuning
More like this →
arXiv AI
Jul 31

Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Models

arXiv:2607. 26119v1 Announce Type: new Abstract: Large reasoning models trained via reinforcement learning (RL) have been increasingly shown to outperform their supervised fine-tuned (SFT) counterparts on mathematical reasoning tasks; Yet the mechanistic basis for this advantage remains unclear.

By Antyabha Rahman, Akshaj Gurugubelli, Omar Ankit, Kevin Zhu, Aishwarya Balwani
reinforcement-learningfine-tuningbenchmarks
More like this →
arXiv Machine Learning
Sep 14

Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models

The paper introduces Decision-Flow Sampling (DF‑Sample), a training‑free, data‑free inference framework that builds a hierarchical reasoning tree, evaluates entire trajectories, and back‑propagates utilities to guide branching decisions. Unlike local step‑wise sampling, DF‑Sample explicitly assesses global paths, enabling it to recover high‑quality, low‑probability reasoning chains that standard decoding misses. On the GPQA benchmark, DF‑Sample attains 45.6% accuracy, outperforming power sampling (38.9%) and GRPO (39.9%) and consistently surpassing baselines across multiple models and benchmarks, demonstrating significant latent reasoning potential in pretrained LLMs.

By Zhendong Mi, Shaoyi Huang
llmsreinforcement-learningfine-tuningbenchmarks
More like this →