arXiv AI
Jul 22
Fluid Reasoning Representations
arXiv:2602. 04843v2 Announce Type: replace Abstract: Frontier large language models increasingly solve complex tasks involving abstract concepts through extended test-time thinking.
Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
The paper investigates how large language models (LLMs) organize reasoning operations—such as problem formulation, goal decomposition, and deduction—within their hidden representation spaces. It shows that these operations are separable in held‑out representations, with peak separability in middle layers, and that token‑wise alignment of operations becomes more distributed across spans as layers deepen. Attention‑masking experiments reveal that representations aligned to operations at chunk onsets depend on prior reasoning context, indicating a geometric correspondence between linguistic reasoning expressions and internal model structure.
Related stories
arXiv AI
Jun 8
A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning
arXiv:2606. 07410v1 Announce Type: cross Abstract: The emergence of "Aha moments" in large language models, particularly DeepSeek-R1-0120, has raised the question of whether these systems genuinely reason or merely imitate the appearance of reasoning.
arXiv AI
Jun 4
From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models
arXiv:2606. 04381v1 Announce Type: cross Abstract: Recent large language models (LLMs) often appear to exhibit spatial reasoning ability; however, this capability is largely \emph{symbolic}, arising from pattern matching over spatial language rather than true \emph{geometric} reasoning over space.
arXiv AI
6d ago
Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Models
The paper introduces LIRSeg, a method that replaces explicit Chain-of-Thought reasoning in multimodal large language models with a compact set of learnable latent tokens for reasoning segmentation. LIRSeg is trained in two stages—spatial alignment and GRPO—while employing extreme-advantage sampling, decoupled exploration-stability updates, and latent diversity amplification to enhance token informativeness. Experiments show that LIRSeg improves segmentation accuracy and reasoning efficiency, achieving significant gIoU gains over the VisionReasoner baseline and reducing reasoning tokens by about 16×.
arXiv Computation and Language
Sep 1
Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
arXiv:2505.16782v3 Announce Type: replace Abstract: Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT...
arXiv AI
Jun 10
Rotate2Think: Geometric Priming via Orthogonal Rotation to Improve Language Model Reasoning
arXiv:2606. 09873v1 Announce Type: cross Abstract: Reasoning models achieve strong performance on challenging tasks by generating explicit intermediate reasoning traces before producing a final answer.
arXiv AI
Sep 10
A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM
A*-Thought-V2 is a framework that models Chain-of-Thought reasoning as a geometric trajectory in a 3D PCA space, using explicit-implicit latent tokens to compress steps that deviate from the main question-to-solution direction. The method measures alignment angles to decide which steps remain text and which become latent, and introduces stepwise embedding forcing and label forcing to train the architecture. Experiments on Qwen models show up to 2.6% accuracy gains, halved response length, and significant reductions in computation and training time.
Hugging Face Trending Papers
Jun 3
From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models
Recent large language models (LLMs) often appear to exhibit spatial reasoning ability; however, this capability is largely \emph{symbolic}, arising from pattern matching over spatial language rather than true \emph{geometric} reasoning over space. Because LLMs operate on discrete tokens, they lack native support for continuous spatial representations, explicit geometric computation, and structured spatial operators.
arXiv AI
Jun 16
RoboPIN: Grounded Embodied Reasoning via Pinned Chain-of-Thought
arXiv:2606. 15753v1 Announce Type: new Abstract: Embodied reasoning requires models to perceive task-relevant objects and spaces in physical environments and maintain consistent visual grounding throughout multi-step reasoning.
arXiv AI
Sep 1
SHAPE of Chain-of-Thought in Math Reasoning
arXiv:2608.28600v1 Announce Type: new Abstract: Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet the mathematically meaningful skills underlying their...
arXiv AI
Jul 28
The Lattice Representation Hypothesis of Large Language Models
arXiv:2603. 01227v3 Announce Type: replace Abstract: We propose the Lattice Representation Hypothesis of large language models: a symbolic backbone that grounds conceptual hierarchies and logical operations in embedding geometry.
arXiv Machine Learning
Aug 27
AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models
The paper introduces AlgoTrace, a framework that traces and steers algorithmic operations in large language models’ latent space during multi‑step reasoning. By clustering latent activations on tasks such as TSP, 3SAT, AIME, and Graph Navigation, the authors identify reusable primitive vectors that can be injected to elicit specific algorithmic behaviors, composed algebraically, and transferred across models and tasks. Fine‑tuning further improves the composition of these primitives, suggesting that LLM reasoning can be viewed as a walk through algorithmic primitives governed by compositional geometry.