arXiv Machine Learning July 27, 2026 By Umberto Biccari, Qian Huang, Enrique Zuazua

Interior interpretability with attention rollout: contraction and propagation profiles in Transformers

Read the original on arXiv Machine Learning →

arXiv:2607. 22367v1 Announce Type: new Abstract: Feature-attribution methods assign scores relating input variables to a model's output, but do not by themselves characterize how explicitly defined interaction operators compose across its intermediate layers.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 3

Attribution via Distributional Paths for Information Revelation

arXiv:2606. 03885v1 Announce Type: new Abstract: Feature attribution methods explain predictions by assigning importance scores to input features.

By Kieran A. Murphy, Shameen Shrestha

arXiv AI

Jun 2

A Monosemantic Attribution Framework for Stable Interpretability in Clinical Neuroscience Transformer-Based Language Models

arXiv:2601. 17952v2 Announce Type: replace-cross Abstract: Interpretability remains a key challenge for deploying language models (LM) in clinical settings such as progression diagnosis of Alzheimer disease, where early and trustworthy predictions are essential.

By Michail Mamalakis, Tiago Azevedo, Cristian Cosentino, Chiara D'Ercoli, Subati Abulikemu, Zhongtian Sun, Richard Bethlehem, Pietro Lio

llms rag safety

arXiv Machine Learning

5d ago

Geometric and Behavioral Stratification in Transformer Residual Streams

arXiv:2608. 12447v1 Announce Type: new Abstract: Trained transformer models develop privileged bases: coordinate axes whose statistics differ from the rest of the residual stream.

By Nelson Guda

llms

arXiv Machine Learning

Jun 30

Generalization Analysis of Transformers in Distribution Regression

arXiv:2606. 29256v1 Announce Type: cross Abstract: In recent years, models based on the Transformer architecture have seen widespread applications and have become one of the core tools in the field of deep learning.

By Peilin Liu, Ding-Xuan Zhou

llms nlp fine-tuning

arXiv Machine Learning

Aug 11

A Mechanistic Diagnostic of Rank Collapse in Post-Norm Decoder Transformers

arXiv:2608. 09417v1 Announce Type: new Abstract: Deep decoder-only Transformers often replace the original Post-Norm architecture with Pre-Norm variants because Post-Norm training is highly sensitive to warmup and learning rate under conventional initialization schemes.

By Xingjian Wang, Qingyu Han, Xiaodong Luo, Yin Zhang

llms

arXiv AI

Jul 23

Geometric Attention: A Regime-Explicit Operator Semantics for Transformer Attention

arXiv:2601. 11618v2 Announce Type: replace-cross Abstract: Geometric Attention (GA) specifies an attention layer by four independent inputs: a finite carrier (what indices are addressable), an evidence-kernel rule (how masked proto-scores and a link induce nonnegative weights), a probe family (which observables are treated as admissible), and an anchor/update rule (which representative kernel is selected and how it is applied).

By Luis Rosario Freytes

llms

Related stories

Attribution via Distributional Paths for Information Revelation

A Monosemantic Attribution Framework for Stable Interpretability in Clinical Neuroscience Transformer-Based Language Models

Geometric and Behavioral Stratification in Transformer Residual Streams

Generalization Analysis of Transformers in Distribution Regression

A Mechanistic Diagnostic of Rank Collapse in Post-Norm Decoder Transformers

Geometric Attention: A Regime-Explicit Operator Semantics for Transformer Attention