← Back to all news
arXiv Computation and Language By Nabeelah Faumi, Deshan Sumanathilaka

Mechanistic Failure Modes of Weight Pruning in Transformer Language Models

Read the original on arXiv Computation and Language →

The study investigates how pruning affects internal mechanisms of GPT‑2 by systematically pruning attention heads and MLP neurons using magnitude and random importance criteria at 3%, 50%, and 70% sparsity. Using the indirect object identification (IOI) circuit as a ground truth, the authors measure circuit integrity with activation patching, path patching, and direct logit attribution, revealing two distinct failure modes: magnitude head pruning essentially zeros the circuit, while magnitude neuron pruning at 30% inverts it, causing the model to prefer incorrect names. The results show that conventional output metrics like perplexity can miss significant mechanistic damage, underscoring the need for circuit‑level causal analysis when evaluating compressed language models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
Aug 25

The Communication Map of a Transformer

The paper introduces the Communication Map, a method that charts every potential communication channel in a transformer model using only its weights. It generalizes previous coupling metrics into a single coefficient covering all 18 connection classes, revealing that 70‑89% of head pairs are non‑randomly oriented and identifying strong or avoiding couplings. The authors demonstrate the map’s utility by recovering known induction circuits and uncovering a two‑dimensional stream subspace whose removal eliminates induction capabilities across several models.

By Richard Zhe Wang
llmssafety
More like this →
arXiv Computation and Language
Oct 28

What Survives Distillation? A Mechanistic Interpretability Study of GPT-2 and DistilGPT-2

The paper investigates how knowledge distillation affects the internal mechanisms of language models by comparing GPT‑2 with its six‑layer student, DistilGPT‑2. Using techniques such as linear CKA, logit lens analysis, and attention pattern decomposition, the authors examine two circuits: Indirect Object Identification (IOI) and Greater‑Than (GT). They find that the IOI circuit collapses completely in the distilled model, whereas the GT circuit largely survives, with the student’s layer‑4 head effectively remapping the teacher’s layer‑9 comparison head, achieving 87.7% of the original task performance.

By Nabeelah Faumi, Deshan Sumanathilaka
llmsefficiencysafety
More like this →
arXiv Machine Learning
Aug 27

When Pruning Meets Interpretability: Preserving Sparse Autoencoder Robustness in LLMs

Sparse autoencoders (SAEs) are commonly used to interpret large language models, but their reliability after pruning is unclear. This study shows that pruning’s effect on an SAE is governed by perturbation energy, a covariance-weighted norm, and that magnitude pruning distorts the representation space by ignoring activation geometry. Activation-aware pruning methods such as Wanda and SparseGPT better preserve SAE behavior, and the authors find that middle layers are especially vulnerable, leading them to propose a layer‑wise sparsity allocation that reduces perplexity for a given sparsity level.

By Suchit Gupte, Xueru Zhang, Mohammad Mahdi Khalili
llmsefficiencysafety
More like this →