arXiv Machine Learning August 10, 2026 By Nikolay Yudin, Sergei Kudriashov, Alexander Gaponov, Maxim Rakhuba

Pay Attention to Attention Distribution: A New Local Lipschitz Bound for Transformers

Read the original on arXiv Machine Learning →

arXiv:2507. 07814v2 Announce Type: replace Abstract: We introduce a novel upper bound on the local Lipschitz constant of the dot-product self-attention block showing its dependence on the attention map distributions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Aug 18

Exact Attention Sensitivity and the Geometry of Transformer Stability

arXiv:2602. 18849v2 Announce Type: replace-cross Abstract: We develop a sensitivity analysis for transformer attention in a geometry aligned with tokenwise computation.

By Seyed Morteza Emadi

llms

arXiv Machine Learning

Jun 17

Softmax as Linear Attention in the Large-Prompt Regime: a Measure-based Perspective

arXiv:2512. 11784v2 Announce Type: replace Abstract: Softmax attention is a central component of transformer architectures, yet its nonlinear structure poses significant challenges for theoretical analysis.

By Etienne Boursier, Claire Boyer

llms

arXiv Machine Learning

Jun 5

Specialization of softmax attention heads: insights from the high-dimensional single-location model

arXiv:2603. 03993v2 Announce Type: replace Abstract: Multi-head attention enables transformer models to represent multiple attention patterns simultaneously.

By M. Sagitova, O. Duranthon, L. Zdeborov\'a

llms

arXiv AI

Aug 24

Jacobian-guided Noise Injection for Quantization Robustness in Large Language Models

arXiv:2608.20988v1 Announce Type: cross Abstract: Quantization of Large Language Models (LLMs) is often hindered by the sensitivity of the self-attention mechanism to discretization errors. We identi...

By Deepanshu Pandey, Arnav Chavan, Nahush Lele, Sankalp Dayal, Deepak Gupta

llms efficiency benchmarks

arXiv Machine Learning

Aug 11

Training-Free Universal Approximation by Prompting Random Transformers

arXiv:2608. 09558v1 Announce Type: new Abstract: How expressive is prompting a transformer?

By Alexander Hsu, Rongjie Lai

llms

arXiv AI

Jun 8

Limitations of Normalization in Attention Mechanism

arXiv:2508. 17821v3 Announce Type: replace-cross Abstract: This paper investigates the limitations of the normalization in attention mechanisms.

By Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova, Radu State

llms

One email a morning, machine-written

Related stories

Exact Attention Sensitivity and the Geometry of Transformer Stability

Softmax as Linear Attention in the Large-Prompt Regime: a Measure-based Perspective

Specialization of softmax attention heads: insights from the high-dimensional single-location model

Jacobian-guided Noise Injection for Quantization Robustness in Large Language Models

Training-Free Universal Approximation by Prompting Random Transformers

Limitations of Normalization in Attention Mechanism