Hugging Face Trending Papers July 20, 2026

L1 Augmented Attention as an Improved Vector Similarity Metric

Read the original on Hugging Face Trending Papers →

Scaled dot product attention conflates directional alignment and vector magnitude, limiting its effectiveness as a similarity metric in Transformer models. We introduce L1 augmented attention, a simple and computationally parallelizable modification that subtracts a learned, head specific L1 distance between queries and keys from the dot product score.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Jul 21

L1 Augmented Attention as an Improved Vector Similarity Metric

arXiv:2607. 18027v1 Announce Type: new Abstract: Scaled dot product attention conflates directional alignment and vector magnitude, limiting its effectiveness as a similarity metric in Transformer models.

By Kurt Godden

llms safety

arXiv Machine Learning

Jun 5

Exact Linear Attention

arXiv:2605. 18848v3 Announce Type: replace Abstract: This paper introduces Exact Linear Attention (ELA), a mechanism that achieves linear computational complexity for Transformer attention by exploiting the exact decomposition property of kernel functions, thereby eliminating approximation error.

By Weinuo Ou

llms computer-vision reinforcement-learning efficiency safety

arXiv AI

Jun 29

IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models

arXiv:2604. 00757v2 Announce Type: replace-cross Abstract: Large Vision Language Models show impressive performance across image and video understanding tasks, yet their computational cost grows rapidly with the number of visual tokens.

By Dong-Jae Lee, Sunghyun Baek, Junmo Kim

llms efficiency

arXiv Machine Learning

Jun 18

Complementary Attention Head Pruning for Efficient Transformers

arXiv:2606. 19150v1 Announce Type: new Abstract: The remarkable success of Transformer-based models in natural language processing stems from architectural scaling, which leads to a large number of parameters and hinders deployment in resource-constrained environments.

By Yaniv Livertovsky, Shahar Somin, Gonen Singer

llms nlp efficiency benchmarks safety

arXiv Machine Learning

Jul 3

Transformer Geometry Observatory TGO-II: Representational Similarity Observatory

arXiv:2607. 02386v1 Announce Type: cross Abstract: While Vision Transformers have achieved remarkable success across computer vision and language applications, the geometric evolution of their internal representations throughout training remains insufficiently understood.

By Kaustubh Kapil, Kishor P. Upla

llms computer-vision safety

arXiv Machine Learning

Jun 5

GRAMformer: Any-Order Modality Interactions via Volumetric Multimodal Cross-Attention

arXiv:2606. 06249v1 Announce Type: cross Abstract: Transformer-based multimodal models rely on attention mechanisms to integrate information across heterogeneous modalities.

By Giordano Cicchetti, Eleonora Grassucci, Danilo Comminiello

llms multimodal

Related stories

L1 Augmented Attention as an Improved Vector Similarity Metric

Exact Linear Attention

IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models

Complementary Attention Head Pruning for Efficient Transformers

Transformer Geometry Observatory TGO-II: Representational Similarity Observatory

GRAMformer: Any-Order Modality Interactions via Volumetric Multimodal Cross-Attention