arXiv Machine Learning July 21, 2026 By Cheng Huan, Hongwei Yuan

An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

Read the original on arXiv Machine Learning →

arXiv:2607. 17696v1 Announce Type: cross Abstract: We develop an adjoint-sensitivity framework for positional influence in causal residual Transformers and separate unconditional analytic results from conditional boundary-shape conclusions.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

1d ago

Exact Attention Sensitivity and the Geometry of Transformer Stability

arXiv:2602. 18849v2 Announce Type: replace-cross Abstract: We develop a sensitivity analysis for transformer attention in a geometry aligned with tokenwise computation.

By Seyed Morteza Emadi

llms

arXiv AI

Jun 12

Transformer Field Theory: A Response-Theoretic Approach to Mechanistic Interpretability

arXiv:2605. 25225v2 Announce Type: replace-cross Abstract: Mechanistic interpretability often studies Transformer behavior by intervening on internal activations through activation patching, causal tracing, path patching, and steering directions.

By David N. Olivieri, Antonio F. P\'erez Rodr\'iguez

llms safety

arXiv AI

Aug 5

Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model

arXiv:2608. 03629v1 Announce Type: new Abstract: A companion paper studies when activation patching and weight-space ablation agree, inside an idealized model where a conditional computation is carried additively through a residual stream.

By Abdallah Khemais

llms benchmarks

arXiv AI

Jun 26

CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry

arXiv:2606. 26538v1 Announce Type: cross Abstract: Deep Transformers are composed of uniformly stacked residual blocks, yet their deepest layers often add little value.

By Huzama Ahmad, Cao Viet Hai Nam, Se-Young Yun

llms efficiency

arXiv AI

Aug 5

A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation

arXiv:2608. 03620v1 Announce Type: cross Abstract: Activation patching and weight-space ablation both claim a component is causally responsible for a behavior, yet they act on different objects: one forward pass versus the parameters behind every forward pass.

By Abdallah Khemais

llms benchmarks

arXiv Machine Learning

6d ago

Orientation, not magnitude: the causal structure of task-vector interference in merged language models

arXiv:2608. 11797v1 Announce Type: new Abstract: Model merging by task arithmetic works until it doesn't, and the field diagnoses why with magnitudes: layerwise representation bias, deviations from cross-task linearity, parameter overlap.

By Chencheng Zhu

llms efficiency safety

Related stories

Exact Attention Sensitivity and the Geometry of Transformer Stability

Transformer Field Theory: A Response-Theoretic Approach to Mechanistic Interpretability

Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model

CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry

A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation

Orientation, not magnitude: the causal structure of task-vector interference in merged language models