arXiv AI August 5, 2026 By Abdallah Khemais

Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model

arXiv:2608. 03629v1 Announce Type: new Abstract: A companion paper studies when activation patching and weight-space ablation agree, inside an idealized model where a conditional computation is carried additively through a residual stream.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Aug 5

A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation

arXiv:2608. 03620v1 Announce Type: cross Abstract: Activation patching and weight-space ablation both claim a component is causally responsible for a behavior, yet they act on different objects: one forward pass versus the parameters behind every forward pass.

By Abdallah Khemais

llms benchmarks

arXiv AI

Jul 21

Every Component is a Lookup: Token Attribution and Composition from a Single Decomposition

arXiv:2605. 23393v2 Announce Type: replace-cross Abstract: Mechanistic interpretability of transformers requires identifying not just which components matter but how they compose into the computational route that produced a prediction.

By Po-Kai Chen, Aske Plaat, Niki van Stein

llms safety

arXiv AI

1d ago

Exact Attention Sensitivity and the Geometry of Transformer Stability

arXiv:2602. 18849v2 Announce Type: replace-cross Abstract: We develop a sensitivity analysis for transformer attention in a geometry aligned with tokenwise computation.

By Seyed Morteza Emadi

llms

arXiv AI

Jul 1

Protein contacts are already in the attention: a single-forward-pass alternative to the Categorical Jacobian

arXiv:2606. 21876v2 Announce Type: replace-cross Abstract: The Categorical Jacobian of Zhang et al.

By Rome Thorstenson

llms

arXiv Machine Learning

Jul 30

A Compositional Theory of Causally Masked Transformers

arXiv:2607. 26988v1 Announce Type: cross Abstract: What types of decision problems can a causally masked, finite-precision transformer solve for inputs of arbitrary length?

By Franz Nowak, Ryan Cotterell, Reda Boumasmoud

llms rag

arXiv AI

1d ago

Gathered, Not Admitted: How Attention Brings a Latent Variable into Verbalizable Form

arXiv:2608. 15022v1 Announce Type: new Abstract: Language models hold latent quantities in a form they can report on, and more of a quantity is present in that form when the task requires reusing it flexibly.

By Parsa Mazaheri

llms benchmarks

Related stories

A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation

Every Component is a Lookup: Token Attribution and Composition from a Single Decomposition

Exact Attention Sensitivity and the Geometry of Transformer Stability

Protein contacts are already in the attention: a single-forward-pass alternative to the Categorical Jacobian

A Compositional Theory of Causally Masked Transformers

Gathered, Not Admitted: How Attention Brings a Latent Variable into Verbalizable Form