arXiv Machine Learning

Are Causal Neuron Circuits Unique and Transferable?

The paper investigates whether groups of neurons that influence a model’s answer are uniquely necessary and transferable across inputs. By exhaustively testing all 4,096 subsets of 12 MLP neurons in Qwen3 and other models, the authors find that only a few neurons are often sufficient to alter predictions, that many subsets can independently flip answers, and that the effect is strongest on the same input chain but weaker on different chains. The study shows that causal neuron circuits are not uniquely defined and that their influence can vary with context and model architecture.

arXiv Computation and Language
Sep 4

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

The study investigates the consistency and specificity of language model circuits across six tasks and five models, focusing on component-level (attention heads and MLP blocks) and neuron-level circuits. Component-level circuits are highly consistent and causally important but lack task specificity, as ablating a circuit for one task similarly harms performance on other tasks. Neuron-level circuits show higher task specificity but lower consistency, with overlap mainly between closely related tasks. The analysis of Llama‑3.2‑3B reveals that shared components are predominantly MLP blocks, while attention heads act as generic attention‑sink heads.

By Michael Li, Nishant Subramani
arXiv AI
Sep 24

Are Stated Reasoning Steps Causally Load-Bearing?

The study investigates whether the reasoning steps a language model writes are causally responsible for its answers. Using a causal intervention method on the activation stream, the authors find that for Qwen3-4B, about 77% of stated steps are causally load‑bearing, while behavioral tests overestimate this by roughly 11 percentage points. The faithfulness of reasoning decreases with model size and depth of reasoning, especially for the smaller Qwen3-1.7B.

By Abhiram Bhupatiraju, Rayan Nyaupane
arXiv Computation and Language
Sep 25

What a Cross-Model Fixed-Point Census Can and Cannot Arbitrate About Repetition

The paper investigates neural text degeneration by measuring the fixed‑point structure of short‑window argmax maps across 17 pretrained models, using 96 random two‑token starts without prompts. It finds a stable four‑way classification that varies across model families and scales, with some models funneling to a single endpoint token while others do not, and shows that this behavior is not solely determined by training data or corpus frequency. The study demonstrates that repetition phenomena are not uniformly explained by either training data or network architecture alone, highlighting the complexity of neural text generation dynamics.

By Nicol\'as Vera Z\'u\~niga
arXiv Computation and Language
Nov 9

Representational drift in a code model: behaviour settles, which neurons carry it does not

The study investigates representational drift in the OpenCoder‑1.5B language model by tracking how neurons encode 169 Python constructs during pretraining on two trillion tokens. While the loss for 16 grammar‑fixing constructs quickly reaches a floor, the specific neurons that carry these constructs continue to change throughout training, with no stable carrier set emerging even at the final checkpoint. The authors find that removing the identified carriers incurs a measurable performance penalty, yet the carrier sets remain dynamic, challenging the hypothesis that they settle over time.

By Piotr Wilam
arXiv Machine Learning
6d ago

When Does the Real Wiring Win? Testing connectome structure-function claims against unmeasured physiology

The paper investigates whether real neural wiring diagrams truly confer functional advantages over shuffled or rewired alternatives by disentangling the effects of model tuning and physiology. It introduces three estimands—physiology tuned on real wiring transferred to nulls (P1), wiring‑specific tuning (P2), and P2 constrained by independent physiological data (P3)—and applies them to synthetic data and three Drosophila circuit models. The study finds that some structure‑function claims hold across all protocols (e.g., segregated dopamine pathways and head‑direction topography), while others fail when realistic physiology is enforced, revealing that apparent wiring advantages may stem from model tuning rather than true biological superiority.

By Yeyuan Liu
arXiv AI
Jul 17

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

arXiv:2607. 14552v1 Announce Type: cross Abstract: A standard recipe for distilling the reasoning ability of large language models (LLMs) is to sample chains of thought from the model, keep those that reach the correct final answer, and fine-tune on the survivors.

By Jungseob Lee, Seungyoon Lee, Suhyune Son, Dongyub Jude Lee, Sungbin Han, Sugyeong Eo, Heuiseok Lim
arXiv Machine Learning
Aug 28

Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit

The paper introduces Circuit Condensation, a post‑training method that prunes low‑attribution edges from large causal graphs and trains a low‑rank adapter to preserve behavior. Across four behaviors and eight models, the condensed circuits are on average 8.1× smaller than the strongest frozen baseline, with reductions up to 316×. Experiments show that weight updates drive the size reduction, and detailed ablations reveal dependencies among remaining edges and a more focused set of heads for indirect object identification.

By Sai Adith Senthil Kumar
arXiv AI
Nov 6

ApproxBrain: Certified Sufficiency Frontiers in Connectome-Constrained Neural Systems

The paper introduces a framework for defining and certifying computational sufficiency in neural systems constrained by a connectome. It formalises sufficiency as a cost-indexed family of neuron subsets conditioned on a task and a tolerance pair, and certifies candidate circuits using one-sided bounds over held-out parameter draws. Experiments on a walking simulator demonstrate that the published 148-neuron circuit is uniquely sufficient, while variations in task or tolerance yield different sufficient subsets, highlighting that sufficiency is set-valued and task-dependent.

By Ali Mahdavi, Azadeh Zamanifar, Amirfarhad Farhadi, Omid Kashefi