arXiv Machine Learning June 5, 2026 By Yongzhong Xu

Pattern Selectivity is Not Task-Causal Structure: A Cross-Architecture Mechanistic Study of Composed-Task Circuits in 1B-Class Language Models

Read the original on arXiv Machine Learning →

arXiv:2606. 05378v1 Announce Type: new Abstract: We test whether a single screen-and-ablate recipe -- identify attention-head circuits by task-pattern selectivity, then verify by causal ablation against a matched-random null -- produces consistent mechanistic claims across model families.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 5

Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

arXiv:2605. 24059v2 Announce Type: replace Abstract: We present a three-step recipe for identifying attention-head circuits in pretrained transformers.

By Yongzhong Xu

llms benchmarks

arXiv AI

Jun 2

When Do Attention Circuits Form? Developmental Trajectories of Capability and Attention-Sink Emergence Across Three 1B-ClassArchitectures

arXiv:2606. 02378v1 Announce Type: cross Abstract: We track the developmental trajectory of attention-head circuit formation across three 1B-class language models spanning two architecture families (dense transformer, mixture-of-experts) and two pretraining corpora (The Pile, DCLM): Pythia 1B, OLMo 1B-0724-hf, and OLMoE 1B-7B-0924.

By Yongzhong Xu

llms safety

arXiv AI

Jun 2

Detection vs. Execution: Single-Bucket Probes Miss Half the Mamba-2 State Sink

arXiv:2606. 00930v1 Announce Type: cross Abstract: Mechanistic interpretability often assumes that probes identifying a representational signature also identify the circuit executing the corresponding computation.

By Yuhang Jiang

benchmarks safety

arXiv Machine Learning

6d ago

Orientation, not magnitude: the causal structure of task-vector interference in merged language models

arXiv:2608. 11797v1 Announce Type: new Abstract: Model merging by task arithmetic works until it doesn't, and the field diagnoses why with magnitudes: layerwise representation bias, deviations from cross-task linearity, parameter overlap.

By Chencheng Zhu

llms efficiency safety

arXiv AI

Aug 6

Necessary, Decodable and Reversible, Yet Not Transferable: A Stress Test for Attention-Head Role Claims

arXiv:2606. 08292v2 Announce Type: replace Abstract: Mechanistic studies often assign a component a role when removing it damages a behavior, its activation linearly encodes task information, and restoring that activation repairs the damage.

By Philip Quirke

arXiv Machine Learning

Jul 27

Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets

arXiv:2607. 21692v1 Announce Type: new Abstract: Sparse attention reduces the cost of long contexts by allowing each query to read only selected parts of the input.

By Jim Allchin

Related stories

Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

When Do Attention Circuits Form? Developmental Trajectories of Capability and Attention-Sink Emergence Across Three 1B-ClassArchitectures

Detection vs. Execution: Single-Bucket Probes Miss Half the Mamba-2 State Sink

Orientation, not magnitude: the causal structure of task-vector interference in merged language models

Necessary, Decodable and Reversible, Yet Not Transferable: A Stress Test for Attention-Head Role Claims

Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets