arXiv Machine Learning July 28, 2026 By Hao Ye (Xi'an Institute of Optics,Precision Mechanics, Chinese Academy of Sciences, University of Chinese Academy of Sciences)

Self-Attention Dynamics with Rotary Position Embeddings: Twisted States and Explicit Consensus Rates on the Sphere

Read the original on arXiv Machine Learning →

arXiv:2607. 24502v1 Announce Type: cross Abstract: Rotary position embeddings (RoPE) modify attention scores through position-dependent rotations, but their effect on normalized token dynamics is not captured by the vanilla spherical self-attention model.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

2d ago

Exact Attention Sensitivity and the Geometry of Transformer Stability

arXiv:2602. 18849v2 Announce Type: replace-cross Abstract: We develop a sensitivity analysis for transformer attention in a geometry aligned with tokenwise computation.

By Seyed Morteza Emadi

llms

arXiv Machine Learning

Jul 9

Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention

arXiv:2607. 06621v1 Announce Type: new Abstract: The pre-softmax score of an attention head is a bilinear form $score(i,j) = x_i^T M x_j$ in a learned operator $M = W_q^T W_k$.

By Li Hengyu (Institute for Solid State Physics, The University of Tokyo)

arXiv Machine Learning

Jun 25

Why Do Accumulated Transformations Extrapolate?

arXiv:2606. 24975v1 Announce Type: new Abstract: PaTH Attention showed that replacing RoPE's position-indexed rotations with accumulated data-dependent Householder reflections yields strong length extrapolation, though performance degrades at extreme context lengths.

By Mahesh Godavarti

arXiv Machine Learning

Jun 11

Attention by Synchronization in Coupled Oscillator Networks

arXiv:2606. 12059v1 Announce Type: new Abstract: We address transformer attention on energy-constrained physical substrates.

By Fabio Pasqualetti, Taosha Guo

llms

arXiv Machine Learning

Jun 3

Trajectory-Aware Node Contributions and the Limits of Static Controllability

arXiv:2606. 03067v1 Announce Type: cross Abstract: A recurring data mining task in complex networks is to determine how individual nodes contribute to system behavior.

By Valentina Kuskova, Dmitry Zaytsev, Michael Coppedge

arXiv Machine Learning

Jul 15

Semidirect Fourier Delta Attention: Phase-Controlled Delta Memory with Constructive Chunk-WY Kernels

arXiv:2607. 11897v1 Announce Type: new Abstract: Linear attention replaces softmax attention's growing KV cache with a fixed recurrent state, but this compression limits exact state tracking and long-context memory.

By Tiantian Zhang

efficiency

Related stories

Exact Attention Sensitivity and the Geometry of Transformer Stability

Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention

Why Do Accumulated Transformations Extrapolate?

Attention by Synchronization in Coupled Oscillator Networks

Trajectory-Aware Node Contributions and the Limits of Static Controllability

Semidirect Fourier Delta Attention: Phase-Controlled Delta Memory with Constructive Chunk-WY Kernels