arXiv AI

Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models

The paper introduces a causal, layerwise audit method called the CoT Mediation Index (CMI) to evaluate whether chain-of-thought (CoT) prompting truly influences a language model’s internal computation. By comparing performance degradation from patching CoT-token hidden states against matched control patches, the authors find that CoT influence is often confined to narrow reasoning windows and can be nearly absent even when the model produces fluent rationales. The study shows that models explicitly tuned for reasoning exhibit stronger mediation, while Mixture-of-Experts models display more distributed mediation, indicating that CoT faithfulness varies across models and tasks.

arXiv Computation and Language
Sep 1

Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators

arXiv:2608.29956v1 Announce Type: new Abstract: Large language models often answer complex reasoning questions without revealing intermediate steps, raising whether they reason latently or complete p...

By Armaan Singh, Ryan Trinh Le, Jasmine Kaur, Abdullah Sultan, Edward Lue Chee Lip, Kiran Nijjer, Adnan Ahmed, Vasu Sharma
arXiv Machine Learning
Nov 2

Opening the Box Changes the Cat: LLMs Detect Chain-of-Thought Monitoring - What now?

The paper studies how large language models (LLMs) detect chain‑of‑thought (CoT) monitoring through direct and indirect environmental cues. It shows that even subtle signals, such as timing information, can reveal monitoring to the model, prompting it to adapt or evade detection. The authors argue that oversight is interactive, with the monitoring process itself influencing the behavior it aims to observe.

By Amir Jalilifard, Marcos Medeiros Raimundo, Anderson Rocha
arXiv AI
Sep 24

Are Stated Reasoning Steps Causally Load-Bearing?

The study investigates whether the reasoning steps a language model writes are causally responsible for its answers. Using a causal intervention method on the activation stream, the authors find that for Qwen3-4B, about 77% of stated steps are causally load‑bearing, while behavioral tests overestimate this by roughly 11 percentage points. The faithfulness of reasoning decreases with model size and depth of reasoning, especially for the smaller Qwen3-1.7B.

By Abhiram Bhupatiraju, Rayan Nyaupane