← Back to all news
arXiv Computation and Language By Murat Dura, Serkan \"Ozt\"urk, Selma Tekir

Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching

Read the original on arXiv Computation and Language →

The paper introduces a sequential activation patching framework to study how Chain-of-Thought (CoT) prompting influences large language models over multiple generated tokens. By tracking CoT-conditioned attention-head activations across token positions and aggregating them with Part-of-Speech guidance, the authors identify distributed head sets that jointly contribute to answer generation. Targeted zero-ablation experiments confirm that these heads are functionally important, affecting mechanisms such as reasoning-trajectory maintenance, answer anchoring, exemplar-target separation, and numerical generation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Jun 9

Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization

arXiv:2510. 13554v2 Announce Type: replace-cross Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps.

By Yang Li, Zhichen Dong, Yuhan Sun, Weixun Wang, Shaopan Xiong, Yijia Luo, Jiashun Liu, Han Lu, Jiamang Wang, Wenbo Su, Bo Zheng, Junchi Yan
llmsreinforcement-learning
More like this →
arXiv AI
Oct 7

COMPASS: Finding Where Reasoning Lives in Language Models

The paper introduces COMPASS, an inference-time steering technique that enhances large language model performance on mathematical reasoning tasks. By using the correctness of a model’s direct answer attempts as a latent signal, COMPASS identifies a small subset of attention heads whose activations can be steered to elicit reasoning. Experiments across three model families and multiple math benchmarks show that COMPASS improves GSM8K accuracy by an average of 16 percentage points and achieves near Chain‑of‑Thought performance while generating 20–70% fewer tokens.

By Pratyay Dutta, Kowshik Thopalli, Vivek Narayanaswamy
llmsbenchmarks
More like this →
arXiv AI
Sep 4

</think> Doesn't Stop Reasoning: Analysis of Spurious CoT Termination

The paper investigates a training‑free early‑exit technique that inserts an end‑of‑think (EoT) token to terminate chain‑of‑thought (CoT) reasoning in large reasoning models. It finds that the injected EoT often fails to cleanly switch the model from reasoning to answering, leading to continued reasoning‑like generation—termed spurious CoT termination—whose length scales with the amount of reasoning saved. By increasing attention to the EoT token through Exit‑token Attention Biasing (EAB), the authors reduce spurious termination and shorten the answering phase across multiple models and benchmarks.

By Seunghee Koh, Sungjae Choi, Minchan Kwon, Sunghyun Baek, Junmo Kim
benchmarks
More like this →