The study investigates the consistency and specificity of language model circuits across six tasks and five models, focusing on component-level (attention heads and MLP blocks) and neuron-level circuits. Component-level circuits are highly consistent and causally important but lack task specificity, as ablating a circuit for one task similarly harms performance on other tasks. Neuron-level circuits show higher task specificity but lower consistency, with overlap mainly between closely related tasks. The analysis of Llama‑3.2‑3B reveals that shared components are predominantly MLP blocks, while attention heads act as generic attention‑sink heads.
By Michael Li, Nishant Subramani
S^3martCirc is a self‑supervised framework that jointly discovers and interprets neural circuits in large language models, rather than treating circuit discovery and functional interpretation as separate stages. It abstracts node behavior into two general computational roles that generalize across tasks and introduces a quantitative metric for assigning these roles, enabling simultaneous identification of importance and function. Experiments demonstrate that S^3martCirc outperforms existing methods in circuit discovery.
By Wendy Zheng, Yinhan He, Liang Wu, Jundong Li
arXiv:2607. 07316v1 Announce Type: new Abstract: This article offers a comprehensive overview of mechanistic interpretability, an emerging field that seeks to reverse-engineer the internal algorithms of modern neural networks.
By Pranav Sawant, Jakub Krej\v{c}\'i
arXiv:2607. 00089v1 Announce Type: new Abstract: Mechanistic interpretability has produced a rich inventory of component-level analyses that characterise what neural-network components encode and how they interact.
By Hussein Chouman, Wataru Sasaki, Tomokazu Matsui, Hirohiko Suwa, Keiichi Yasumoto
arXiv:2601. 09624v2 Announce Type: replace-cross Abstract: Machine unlearning is becoming essential for building trustworthy and compliant language models.
By Jiali Cheng, Ziheng Chen, Chirag Agarwal, Hadi Amiri
arXiv:2609.36813v1 Announce Type: new
Abstract: Large language models (LLMs) exhibit strong general capabilities that mechanistic interpretability has attributed to sparse computational circuits. How...
By Chuanpu Liu, Miao Yu, Yikai Cai, Yuanhe Zhang, Zhenhong Zhou, Li Sun, Zuming Jiang, Yufei Guo