arXiv AI
The Misery of Mechanistic Interpretability: A Formal Perspective
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Oct 7
Weight Oracles: Reading Neural Network Weights with Language Models
arXiv:2610.07334v1 Announce Type: new Abstract: Interpretability methods for neural networks are predominantly reactive: they analyse activations produced during specific forward passes, requiring kn...
arXiv Machine Learning
Jul 2
Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol
arXiv:2607. 00089v1 Announce Type: new Abstract: Mechanistic interpretability has produced a rich inventory of component-level analyses that characterise what neural-network components encode and how they interact.
arXiv AI
Jul 7
Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models
arXiv:2503. 06269v3 Announce Type: replace-cross Abstract: Traditional white-box methods for creating adversarial perturbations against LLMs typically rely only on gradient computation from the targeted model, ignoring the internal mechanisms responsible for attack success or failure.
arXiv Machine Learning
Jul 1
Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?
arXiv:2606. 32008v1 Announce Type: new Abstract: Mechanistic interpretability (MI) requires full access to model internals, yet the APIs for most widely deployed language models at best expose log-probabilities over output tokens.
arXiv AI
Oct 16
Interpreting Language Model Parameters
The paper introduces adVersarial Parameter Decomposition (VPD), a method that splits a language model’s parameters into simple subcomponents, each responsible for a small part of the model’s algorithm. VPD optimizes these decompositions so that the model’s input‑output behavior remains intact even when many subcomponents are removed, including adversarially chosen ones. This approach enables a new form of circuit analysis by tracing how sequences of parameter subcomponents interact to produce outputs for specific inputs.
arXiv Machine Learning
Oct 8
U-Space: Uncovering When and Why Uncertainty Arises in Language Models
arXiv:2610.09087v1 Announce Type: cross Abstract: Large language models are informing decisions with ever-higher stakes. As the consequences of their errors grow, a central question becomes harder to...