arXiv Machine Learning

CircuitKIT : Circuit Discovery, Evaluation, and Application Toolkit for Mechanistic Interpretability

arXiv:2607. 19317v1 Announce Type: new Abstract: Circuit analysis can support not only model explanation but also downstream interventions such as pruning, editing, steering, and selective fine-tuning.

arXiv AI
Sep 15

Enhancing Large Language Model-Based Systems for End-to-End Circuit Analysis Problem Solving

The paper introduces an enhanced end‑to‑end circuit analysis framework built on Gemini 2.5 Pro, targeting engineering education. It addresses two key failure modes—circuit‑recognition hallucinations and reasoning‑process hallucinations—by adding a YOLO detector for source polarity re‑identification and an ngspice verification loop for iterative refinement. The resulting pipeline achieves 97.59 % accuracy on 83 undergraduate problems, markedly outperforming the baseline Gemini model and demonstrating significant gains across varied diagram styles and textbooks.

By Liangliang Chen, Weiyu Sun, Huiru Xie, Yongnuo Cai, Ying Zhang
arXiv AI
Sep 2

S^3martCirc: Self-supervised Smart Circuit Discovery

S^3martCirc is a self‑supervised framework that jointly discovers and interprets neural circuits in large language models, rather than treating circuit discovery and functional interpretation as separate stages. It abstracts node behavior into two general computational roles that generalize across tasks and introduces a quantitative metric for assigning these roles, enabling simultaneous identification of importance and function. Experiments demonstrate that S^3martCirc outperforms existing methods in circuit discovery.

By Wendy Zheng, Yinhan He, Liang Wu, Jundong Li
arXiv Computation and Language
Aug 28

Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery

The paper investigates whether structural differences in circuits discovered by circuit discovery methods reflect distinct mechanisms. By varying input-token frequency while keeping the task constant, the authors find that although circuits appear specialized by frequency structurally, functional and representational analyses reveal no reliable differences, a phenomenon they call phantom specialization. Across multiple models and tasks, structurally distinct circuits implement the same computation, with core shared subgraphs recovering most of the performance and interchangeable internal representations confirmed by causal interventions.

By Alireza Bayat Makou, Jingcheng Niu, Subhabrata Dutta, Iryna Gurevych
arXiv Machine Learning
Oct 3

Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability

The paper investigates mechanistic interpretability, focusing on how automated circuit discovery is evaluated. It shows that the commonly used faithfulness objective can favor circuits that reproduce a model’s behavior poorly, creating an objective-level recovery gap. Experiments on four human-reference tasks and InterpBench reveal that many discovery methods misrank candidate circuits, and that restoring excluded signals can correct most of these misrankings without altering the circuits’ behavior.

By Chuqin Geng, Li Zhang, Haolin Ye, Mark Zhang, Luke Zhang, Xujie Si