arXiv:2607. 01553v1 Announce Type: cross Abstract: Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.
arXiv:2604. 17324v2 Announce Type: replace-cross Abstract: Global self-attention drives modern graph transformers, yet the softmax at its core imposes a structural constraint rarely examined directly: every attention row is non-negative and sums to one, so each per-head output is a mass-conserving convex combination of value vectors.
arXiv:2602. 01553v3 Announce Type: replace-cross Abstract: Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.
arXiv:2607. 17272v1 Announce Type: new Abstract: Node representation learning has advanced rapidly, yet most existing methods rely on per-dataset training and hyperparameter tuning.
arXiv:2502. 16533v3 Announce Type: replace-cross Abstract: Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing.
arXiv:2602.15239v3 Announce Type: replace Abstract: Transformers have achieved remarkable success across domains, motivating the rise of Graph Transformers (GTs) as attention-based architectures for...