arXiv AI June 9, 2026 By Yang Liu, Dongxin Guo, Tom Zheng, Siu Ming Yiu, Liam Ning, Jikun Wu

Capacity-Controlled Global Attention for Graph Transformers

arXiv:2604. 17324v2 Announce Type: replace-cross Abstract: Global self-attention drives modern graph transformers, yet the softmax at its core imposes a structural constraint rarely examined directly: every attention row is non-negative and sums to one, so each per-head output is a mass-conserving convex combination of value vectors.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jul 3

X-LogSMask: Expand Transformer for Graph-Structured Data

arXiv:2607. 01553v1 Announce Type: cross Abstract: Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale.

By Leyan Li, Rennong Yang, Zhenxing Zhang, Liping Hu

llms benchmarks safety

arXiv Machine Learning

Jun 18

Complementary Attention Head Pruning for Efficient Transformers

arXiv:2606. 19150v1 Announce Type: new Abstract: The remarkable success of Transformer-based models in natural language processing stems from architectural scaling, which leads to a large number of parameters and hinders deployment in resource-constrained environments.

By Yaniv Livertovsky, Shahar Somin, Gonen Singer

llms nlp efficiency benchmarks safety

arXiv Machine Learning

Jul 31

THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model

arXiv:2607. 27303v1 Announce Type: new Abstract: Temporal heterogeneous graphs offer a natural abstraction for dynamic relational systems in which diverse node and relation types co-exist and evolve over time.

By Yixin Peng, Diego Collarana, Er Jin, Stefan Decker

llms benchmarks

arXiv Machine Learning

Jul 21

A Weisfeiler-Leman Characterization of Global-Attention Graph Transformers for Mixed-Integer Linear Programs

arXiv:2607. 17570v1 Announce Type: new Abstract: Graph foundation models (GFMs) with global attention are increasingly used to represent mixed-integer linear programs (MILPs), aiming to capture structure beyond the locality of standard graph neural networks.

By Md Abrar Jahin, Craig A. Knoblock, Jay Pujara

llms rag

arXiv Machine Learning

Jun 2

Don't Read Everything: A Curvature-Conditioned Query for Linear Attention

arXiv:2606. 01294v1 Announce Type: cross Abstract: Linear attention reduces the quadratic cost of softmax attention by maintaining a recurrent fast-weight state, but it consistently lags on in-context retrieval and long-context tasks.

By Dong Le, Thong Nguyen, Cong-Duy Nguyen, Anh Tuan Luu

arXiv AI

Jul 28

Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models

arXiv:2607. 23054v1 Announce Type: cross Abstract: Multi-head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value pairs through a shared low-rank bottleneck (cKV), achieving 81% KV-cache reduction during inference.

By Dhruvil S, Fenil Sojitra, Ravirajsinh Chauhan

llms fine-tuning safety

Related stories

X-LogSMask: Expand Transformer for Graph-Structured Data

Complementary Attention Head Pruning for Efficient Transformers

THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model

A Weisfeiler-Leman Characterization of Global-Attention Graph Transformers for Mixed-Integer Linear Programs

Don't Read Everything: A Curvature-Conditioned Query for Linear Attention

Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models