arXiv AI
Representational Simplicity and Circuit Size Dissociate in a Threshold-Dependent Way: A Controlled Test via Adversarial Training
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 22
The Information Shadow: Measuring Structural Limits on What Language Models Can Learn
arXiv:2607. 18305v1 Announce Type: cross Abstract: Some limits on what language models know are not gaps in data coverage but structural properties of learning from text.
arXiv Machine Learning
Sep 23
Matryoshka attribution: Learning to attribute language model outputs to representations and weights
Matryoshka Attribution (MAttr) is a mask‑learning method that identifies nested subsets of a language model’s internal components by minimizing downstream loss. It uses a differentiable sigmoid top‑k operator and randomizes sparsity during training to produce an attribution ordering of components. MAttr tops the Mechanistic Interpretability Benchmark leaderboard and can be applied via reinforcement learning to pinpoint weight changes that control behaviors such as refusal in Llama 3.1 8B Instruct, where restoring just 1% of weights removes refusals while preserving capabilities.
arXiv Machine Learning
Aug 31
How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution
The paper introduces Concept-Targeted Attribution (CTA), a method that trains attribution graphs to explain the emergence of internal concept representations in language models, rather than just the final token prediction. CTA produces probe-specific circuits that reveal which internal computations drive a linear probe’s accuracy, and cross-layer transcoders demonstrate that these graphs contain predictive structure across multiple concept categories. Causal ablations show that probe-targeted and logit-targeted graphs capture distinct mechanisms, with probe-relevant features affecting internal concept scores and logit-relevant features altering generated tokens.
arXiv Machine Learning
Sep 16
Test-Time Unlearning via Sparse Autoencoder
arXiv:2609.16229v1 Announce Type: new Abstract: Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify...
arXiv Machine Learning
Aug 28
Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit
The paper introduces Circuit Condensation, a post‑training method that prunes low‑attribution edges from large causal graphs and trains a low‑rank adapter to preserve behavior. Across four behaviors and eight models, the condensed circuits are on average 8.1× smaller than the strongest frozen baseline, with reductions up to 316×. Experiments show that weight updates drive the size reduction, and detailed ablations reveal dependencies among remaining edges and a more focused set of heads for indirect object identification.
arXiv Machine Learning
Jun 2
Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention
arXiv:2605. 29548v2 Announce Type: replace Abstract: Larger models learn tasks smaller models do not.