arXiv AI
Aug 11
The Transformer Revolution, Part 1: Dynamic Processing through Output-Weight Interconnections
arXiv:2608. 03921v2 Announce Type: replace Abstract: This paper offers a new interpretation of the Transformer during inference.
Interpreting Language Model Parameters
The paper introduces adVersarial Parameter Decomposition (VPD), a method that splits a language model’s parameters into simple subcomponents, each responsible for a small part of the model’s algorithm. VPD optimizes these decompositions so that the model’s input‑output behavior remains intact even when many subcomponents are removed, including adversarially chosen ones. This approach enables a new form of circuit analysis by tracing how sequences of parameter subcomponents interact to produce outputs for specific inputs.
Related stories
arXiv AI
Aug 5
The Transformer Revolution, Part 1: Dynamic Processing through Output- Weight Interconnections
arXiv:2608. 03921v1 Announce Type: new Abstract: This paper offers a new interpretation of the Transformer during inference.
arXiv Machine Learning
Sep 11
Perturbation: A simple and efficient adversarial tracer for representation learning in language models
The paper introduces Perturbation, a method that treats representations in language models as learning conduits rather than activation patterns. By fine‑tuning a model on a single adversarial example and observing how this perturbation spreads to other inputs, the approach avoids geometric assumptions and does not identify representations in untrained models. In trained models, Perturbation uncovers structured transfer across multiple linguistic scales, indicating that language models generalize along representational lines and acquire linguistic abstractions through experience.
arXiv AI
Sep 10
Instance-wise Linearization of Neural Network for Model Interpretation
arXiv:2310.16295v2 Announce Type: replace-cross Abstract: Neural network have achieved remarkable successes in many scientific fields. However, the interpretability of the neural network model is sti...
arXiv Machine Learning
Aug 10
Mathematical Principles and Experimental Discoveries of the Emergence of Symbolic Patterns in Artificial Neural Networks
arXiv:2608. 06839v1 Announce Type: new Abstract: Artificial Neural networks (ANNs) are often treated as black-box models, making explainability a central challenge in deep learning.
arXiv AI
Jun 12
Language Model Circuits Are Sparse in the Neuron Basis
arXiv:2601. 22594v2 Announce Type: replace-cross Abstract: The high-level concepts that a neural network uses to perform computation need not be aligned to individual neurons (Smolensky, 1986).
arXiv Machine Learning
Jun 17
A Dynamical Systems Perspective on the Analysis of Neural Networks
arXiv:2507. 05164v2 Announce Type: replace-cross Abstract: In this chapter, we utilize dynamical systems to analyze several aspects of machine learning algorithms.
arXiv AI
Aug 20
Mechanistic Interpretability of Structure-Aware Numerical Reasoning in LLaMA 3.1 8B
The paper investigates how LLaMA 3.1‑8B models numerical sequence patterns, focusing on time‑series prediction. By designing a task that requires detecting structural cues—specifically first differences in a sequence—the authors show that the model performs well and internally computes and stores these differences. Probing and activation‑patching experiments reveal that LLaMA retrieves and applies the first‑difference via an induction‑like circuit, marking one of the first demonstrations of concept induction in large language models.
arXiv Machine Learning
Jun 18
From Mechanistic to Compositional Interpretability
arXiv:2605. 08934v2 Announce Type: replace Abstract: Mechanistic interpretability aims to explain neural model behaviour by reverse-engineering learned computational structure into human-understandable components.
arXiv AI
Jul 8
Transformers converge to invariant algorithmic cores
arXiv:2602. 22600v2 Announce Type: replace-cross Abstract: Training selects for behavior, not circuitry: many weight configurations can implement the same function.
arXiv Machine Learning
Jul 9
Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning
arXiv:2607. 07316v1 Announce Type: new Abstract: This article offers a comprehensive overview of mechanistic interpretability, an emerging field that seeks to reverse-engineer the internal algorithms of modern neural networks.
arXiv AI
Sep 15
The Misery of Mechanistic Interpretability: A Formal Perspective
arXiv:2609.15533v1 Announce Type: cross Abstract: Mechanistic interpretability has become the dominant lens for understanding frontier language models, as their inner workings are complex and inheren...