← Back to all news
arXiv Computation and Language By Seogyeong Jeong, Jaehui Hwang, Dongyoon Han, Geonmo Gu, Alice Oh, Taekyung Kim

Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs

Read the original on arXiv Computation and Language →

The paper investigates how large language models (LLMs) organize reasoning operations—such as problem formulation, goal decomposition, and deduction—within their hidden representation spaces. It shows that these operations are separable in held‑out representations, with peak separability in middle layers, and that token‑wise alignment of operations becomes more distributed across spans as layers deepen. Attention‑masking experiments reveal that representations aligned to operations at chunk onsets depend on prior reasoning context, indicating a geometric correspondence between linguistic reasoning expressions and internal model structure.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Jul 22

Fluid Reasoning Representations

arXiv:2602. 04843v2 Announce Type: replace Abstract: Frontier large language models increasingly solve complex tasks involving abstract concepts through extended test-time thinking.

By Dmitrii Kharlapenko, Terry Jingchen Zhang, Arth Singh, Alessandro Stolfo, Arthur Conmy, Mrinmaya Sachan, Zhijing Jin
llms
More like this →
arXiv AI
Jun 4

From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models

arXiv:2606. 04381v1 Announce Type: cross Abstract: Recent large language models (LLMs) often appear to exhibit spatial reasoning ability; however, this capability is largely \emph{symbolic}, arising from pattern matching over spatial language rather than true \emph{geometric} reasoning over space.

By Chen Chu, Bita Azarijoo, Li Xiong, Khurram Shafique, Cyrus Shahabi
llmsmultimodalbenchmarks
More like this →
arXiv AI
6d ago

Skip the Talk, Re-Focus on Vision: Latent Reasoning for Reasoning Segmentation in Multimodal Large Language Models

The paper introduces LIRSeg, a method that replaces explicit Chain-of-Thought reasoning in multimodal large language models with a compact set of learnable latent tokens for reasoning segmentation. LIRSeg is trained in two stages—spatial alignment and GRPO—while employing extreme-advantage sampling, decoupled exploration-stability updates, and latent diversity amplification to enhance token informativeness. Experiments show that LIRSeg improves segmentation accuracy and reasoning efficiency, achieving significant gIoU gains over the VisionReasoner baseline and reducing reasoning tokens by about 16×.

By Tianhang Guo, Yulin He, Wei Chen, Wenjuan Zhou, Yuhang Li, Xinbiao Gan
llmsagentscomputer-visionroboticsmultimodalbenchmarkssafety
More like this →