arXiv Machine Learning By \"Omer Tekin Yavuz (\.Ileti\c{s}im Yaz{\i}l{\i}m)

Are Causal Neuron Circuits Unique and Transferable?

Read the original on arXiv Machine Learning →

The paper investigates whether groups of neurons that influence a model’s answer are uniquely necessary and transferable across inputs. By exhaustively testing all 4,096 subsets of 12 MLP neurons in Qwen3 and other models, the authors find that only a few neurons are often sufficient to alter predictions, that many subsets can independently flip answers, and that the effect is strongest on the same input chain but weaker on different chains. The study shows that causal neuron circuits are not uniquely defined and that their influence can vary with context and model architecture.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computation and Language
Sep 4

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

The study investigates the consistency and specificity of language model circuits across six tasks and five models, focusing on component-level (attention heads and MLP blocks) and neuron-level circuits. Component-level circuits are highly consistent and causally important but lack task specificity, as ablating a circuit for one task similarly harms performance on other tasks. Neuron-level circuits show higher task specificity but lower consistency, with overlap mainly between closely related tasks. The analysis of Llama‑3.2‑3B reveals that shared components are predominantly MLP blocks, while attention heads act as generic attention‑sink heads.

By Michael Li, Nishant Subramani
arXiv AI
Sep 24

Are Stated Reasoning Steps Causally Load-Bearing?

The study investigates whether the reasoning steps a language model writes are causally responsible for its answers. Using a causal intervention method on the activation stream, the authors find that for Qwen3-4B, about 77% of stated steps are causally load‑bearing, while behavioral tests overestimate this by roughly 11 percentage points. The faithfulness of reasoning decreases with model size and depth of reasoning, especially for the smaller Qwen3-1.7B.

By Abhiram Bhupatiraju, Rayan Nyaupane
arXiv Computation and Language
Sep 25

What a Cross-Model Fixed-Point Census Can and Cannot Arbitrate About Repetition

The paper investigates neural text degeneration by measuring the fixed‑point structure of short‑window argmax maps across 17 pretrained models, using 96 random two‑token starts without prompts. It finds a stable four‑way classification that varies across model families and scales, with some models funneling to a single endpoint token while others do not, and shows that this behavior is not solely determined by training data or corpus frequency. The study demonstrates that repetition phenomena are not uniformly explained by either training data or network architecture alone, highlighting the complexity of neural text generation dynamics.

By Nicol\'as Vera Z\'u\~niga