arXiv AI July 23, 2026 By Luis Rosario Freytes

Geometric Attention: A Regime-Explicit Operator Semantics for Transformer Attention

arXiv:2601. 11618v2 Announce Type: replace-cross Abstract: Geometric Attention (GA) specifies an attention layer by four independent inputs: a finite carrier (what indices are addressable), an evidence-kernel rule (how masked proto-scores and a link induce nonnegative weights), a probe family (which observables are treated as admissible), and an anchor/update rule (which representative kernel is selected and how it is applied).

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

llms

arXiv Machine Learning

Jun 5

Exact Linear Attention

arXiv:2605. 18848v3 Announce Type: replace Abstract: This paper introduces Exact Linear Attention (ELA), a mechanism that achieves linear computational complexity for Transformer attention by exploiting the exact decomposition property of kernel functions, thereby eliminating approximation error.

By Weinuo Ou

llms computer-vision reinforcement-learning efficiency safety

arXiv Machine Learning

Jun 9

A Unifying View of Attention Sinks: Two Algorithms, Two Solutions

arXiv:2606. 08105v1 Announce Type: new Abstract: When attention concentrates on a single token, a sink, what is the model actually computing?

By Lukas Fesser, Mozes Jacobs, Thomas Fel, Andy Keller, Sham Kakade

llms

arXiv Machine Learning

Jul 28

The Entropic Bound for Transformers: Why Static Rank Fails and Attention-Native Rank Recovers

arXiv:2607. 23050v1 Announce Type: new Abstract: Neural scaling laws describe how loss decreases as models, data, and compute grow, but they do not answer a prior question: for a fixed task, what is the minimum model capacity required to solve it?

By Byeong Hoon Yoon

llms safety

arXiv Machine Learning

Jun 19

The Token Is a Group Element: On Lie-Algebra Attention over Matrix Lie Groups

arXiv:2606. 20547v1 Announce Type: new Abstract: We place the attention token on the group: a token is an element $g_i$ of a matrix Lie group $G$ -- a bare transformation, with no feature payload and no external action $\rho(g)$ carrying it.

By Przemyslaw Musialski

arXiv Machine Learning

Jul 3

Transformer Geometry Observatory TGO-II: Representational Similarity Observatory

arXiv:2607. 02386v1 Announce Type: cross Abstract: While Vision Transformers have achieved remarkable success across computer vision and language applications, the geometric evolution of their internal representations throughout training remains insufficiently understood.

By Kaustubh Kapil, Kishor P. Upla

llms computer-vision safety

arXiv Machine Learning

Jul 9

FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention

arXiv:2607. 07478v1 Announce Type: new Abstract: FFT-based spectral preprocessing of learned query-key (Q/K) projections substantially improves transformer attention on character-level language modelling.

By Athanasios Zeris

llms rag

Related stories

Exact Linear Attention

A Unifying View of Attention Sinks: Two Algorithms, Two Solutions

The Entropic Bound for Transformers: Why Static Rank Fails and Attention-Native Rank Recovers

The Token Is a Group Element: On Lie-Algebra Attention over Matrix Lie Groups

Transformer Geometry Observatory TGO-II: Representational Similarity Observatory

FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention