arXiv Machine Learning June 17, 2026 By Etienne Boursier, Claire Boyer

Softmax as Linear Attention in the Large-Prompt Regime: a Measure-based Perspective

Read the original on arXiv Machine Learning →

arXiv:2512. 11784v2 Announce Type: replace Abstract: Softmax attention is a central component of transformer architectures, yet its nonlinear structure poses significant challenges for theoretical analysis.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Aug 11

Training-Free Universal Approximation by Prompting Random Transformers

arXiv:2608. 09558v1 Announce Type: new Abstract: How expressive is prompting a transformer?

By Alexander Hsu, Rongjie Lai

llms

arXiv Machine Learning

Jul 2

Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization

arXiv:2607. 00479v1 Announce Type: new Abstract: Transformer-based large models have demonstrated remarkable generalization abilities across different tasks by leveraging a context-aware attention module for in-context learning.

By Peilin Liu, Ding-Xuan Zhou

llms

arXiv Machine Learning

Jul 28

The Entropic Bound for Transformers: Why Static Rank Fails and Attention-Native Rank Recovers

arXiv:2607. 23050v1 Announce Type: new Abstract: Neural scaling laws describe how loss decreases as models, data, and compute grow, but they do not answer a prior question: for a fixed task, what is the minimum model capacity required to solve it?

By Byeong Hoon Yoon

llms safety

arXiv Machine Learning

Jul 22

Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention

arXiv:2607. 18759v1 Announce Type: new Abstract: Transformers with relative positional encodings often extrapolate to sequences longer than those seen during training, whereas transformers with learned absolute encodings typically do not.

By Subham Singh, Ashutosh Mishra, Subha Raut

llms safety

arXiv Machine Learning

Jun 24

Asymptotic Signal Subspace Recovery in Softmax Attention Models

arXiv:2606. 22406v2 Announce Type: replace Abstract: Attention mechanisms have demonstrated remarkable empirical success in identifying relevant information from large collections of tokens, yet the theoretical principles underlying this behavior remain poorly understood.

By Lan V. Truong

arXiv Machine Learning

Jun 3

Neural Attention Search Linear: Towards Adaptive Token-Level Hybrid Attention Models

arXiv:2602. 03681v2 Announce Type: replace-cross Abstract: The quadratic computational complexity of softmax transformers has become a bottleneck in long-context scenarios.

By Difan Deng, Andreas Bentzen Winje, Lukas Fehring, Marius Lindauer

llms

One email a morning, machine-written

Related stories

Training-Free Universal Approximation by Prompting Random Transformers

Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization

The Entropic Bound for Transformers: Why Static Rank Fails and Attention-Native Rank Recovers

Relative Positions Generalize, Absolute Positions Memorize: An Implicit-Bias Account of Length Generalization in Attention

Asymptotic Signal Subspace Recovery in Softmax Attention Models

Neural Attention Search Linear: Towards Adaptive Token-Level Hybrid Attention Models