The paper investigates whether the tokens highlighted by large language models (LLMs) as key to their predictions are behaviorally consistent when those tokens are manipulated. Using sufficiency, comprehensiveness, and counterfactuality metrics, the authors evaluate closed‑source GPT‑4o‑mini and open‑source models Gemma3, Granite8B, and DeepSeek on IMDB and Steam review datasets. Results show that GPT‑4o‑mini consistently follows expected patterns, Gemma3 and Granite8B maintain sufficiency but falter under stricter interventions, and DeepSeek variants exhibit structural inconsistencies, indicating that explanation reliability varies across model families and sizes.
By Alisson Rosa Pereira, Bruno Iochins Grisci
The paper introduces a method for analyzing how interventions in a prompt affect a language model’s reasoning by using behavioral counterfactuals. It trains a module to edit the residual stream so that the next-token distributions for a fixed response match between a source and a counterfactual prompt, outperforming standard activation reconstruction. The resulting attribution maps and visualizations help identify causally important locations for counterfactual recovery and guide hypothesis testing with activation patching, as demonstrated in a case study on Gemma-4-E4B-it and DeepSeek-R1-0528-Qwen3-8B.
By Shawn Im, James Oldfield, Adel Bibi, Sharon Li
Faithfully understanding the internal process that occurs as models form their chain of thought is an important problem with the rise in model capabilities. However, this remains challenging due to th...
arXiv:2603. 28371v2 Announce Type: replace-cross Abstract: When an agent can articulate why something works, we typically take this as evidence of genuine understanding.
By Camilo Chac\'on Sartori
arXiv:2609.08216v1 Announce Type: new
Abstract: Agentic AI systems built on large language models fail in two persistent ways that scaling does not fix: they break under distribution shift, and they...
By Arun Vignesh Malarkkan, Xinyuan Wang, Yanjie Fu
The paper introduces a refined evaluation framework for chain‑of‑thought (CoT) monitoring in large language models, showing that the commonly used minimal‑criterion sensitivity metric overestimates causal detection of targeted behaviors. It establishes upper and lower bounds on causal detection rates and uses these to assess monitorability. The authors then create a dataset of control–intervention task pairs and apply three training methods—off‑policy self‑distillation, reinforcement learning, and on‑policy self‑distillation—to produce model variants that consistently improve CoT monitorability across various configurations.
By Qiang Fu, Xinyue Li, Andre Wibisono, Arman Cohan