arXiv Machine Learning
Jun 3
Attribution via Distributional Paths for Information Revelation
arXiv:2606. 03885v1 Announce Type: new Abstract: Feature attribution methods explain predictions by assigning importance scores to input features.
How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution
The paper introduces Concept-Targeted Attribution (CTA), a method that trains attribution graphs to explain the emergence of internal concept representations in language models, rather than just the final token prediction. CTA produces probe-specific circuits that reveal which internal computations drive a linear probe’s accuracy, and cross-layer transcoders demonstrate that these graphs contain predictive structure across multiple concept categories. Causal ablations show that probe-targeted and logit-targeted graphs capture distinct mechanisms, with probe-relevant features affecting internal concept scores and logit-relevant features altering generated tokens.
Related stories
arXiv Machine Learning
Sep 23
Matryoshka attribution: Learning to attribute language model outputs to representations and weights
Matryoshka Attribution (MAttr) is a mask‑learning method that identifies nested subsets of a language model’s internal components by minimizing downstream loss. It uses a differentiable sigmoid top‑k operator and randomizes sparsity during training to produce an attribution ordering of components. MAttr tops the Mechanistic Interpretability Benchmark leaderboard and can be applied via reinforcement learning to pinpoint weight changes that control behaviors such as refusal in Llama 3.1 8B Instruct, where restoring just 1% of weights removes refusals while preserving capabilities.
arXiv Machine Learning
Jun 11
Mechanisms of Introspective Awareness
arXiv:2603. 21396v5 Announce Type: replace Abstract: Recent work has shown that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept -- a phenomenon termed "introspective awareness.
arXiv AI
Sep 30
Representational Simplicity and Circuit Size Dissociate in a Threshold-Dependent Way: A Controlled Test via Adversarial Training
arXiv:2609.35890v1 Announce Type: new Abstract: Sparse-autoencoder decomposability and concentrated feature attribution are increasingly treated as evidence that a model's computation is easier to re...
arXiv AI
Jul 21
Every Component is a Lookup: Token Attribution and Composition from a Single Decomposition
arXiv:2605. 23393v2 Announce Type: replace-cross Abstract: Mechanistic interpretability of transformers requires identifying not just which components matter but how they compose into the computational route that produced a prediction.
arXiv AI
Jul 28
A Mechanistic Perspective and Circuit-Guided Difficulty Metric for Unlearning
arXiv:2601. 09624v2 Announce Type: replace-cross Abstract: Machine unlearning is becoming essential for building trustworthy and compliant language models.
arXiv Machine Learning
Oct 23
Rethinking SAE Evaluation: An Atomic Interpretable Unit Framework Reveals Hidden Polysemanticity and Redundancy
The paper introduces a new framework for evaluating sparse autoencoders (SAEs) by requiring their features to form an atomic interpretable unit basis, a coordinate system for semantically meaningful explanation. It formalizes this objective and organizes existing and new metrics, such as DAME, into a comprehensive benchmarking framework. Experiments across many SAE architectures reveal that some models appear monosemantic under current metrics yet are functionally polysemantic, with dominant features being less interpretable and more redundant.
arXiv AI
Jul 22
Circuit Claims Depend on What Is Extracted and How It Is Compared
arXiv:2607. 18921v1 Announce Type: cross Abstract: Circuit extraction identifies a small set of model components whose presence preserves a target behavior under ablation, and the resulting circuit is often read as the mechanism behind that behavior.
arXiv AI
Jun 30
Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies
arXiv:2606. 29171v1 Announce Type: cross Abstract: While existing data attribution methods can identify which training examples build specific mechanistic circuits, they cannot explain how training data shapes the high-level behavioral decisions a model learns to make.
arXiv AI
Sep 25
Every Component Is a Lookup: One Linear Graph for Interaction, Composition and Attribution
The paper proposes that two architectural assumptions—(1) attention and MLPs share a key‑value form <phi(S)>U, and (2) components read from an additive residual stream—are sufficient to answer three interpretability questions: component interaction, information routing, and token attribution. By treating these selections as a computational graph, the authors develop Unpack, a backward attribution method that validates interaction scores, recovered routes, and token attribution against established tests across models ranging from 160M to 6.9B parameters. The study also shows that contribution and causal effect can differ, with a recognizable signature in how components change when a task is removed.
arXiv AI
Oct 15
Data-driven Circuit Discovery for Interpretability of Language Models
arXiv:2605.09129v2 Announce Type: replace Abstract: Circuit study aims to explain how language models (LMs) implement a task by localizing and interpreting a circuit: a computational subgraph respons...
arXiv Machine Learning
Sep 22
Circuit-Diff: Factual Edit-based Intervention Method for Localizing Knowledge in Attribution Graphs
arXiv:2609.23892v1 Announce Type: new Abstract: Mechanistic interpretability defines features as the fundamental units of a neural network and circuits as the weighted subgraphs that carry out its co...