Graph Classification with Transformers

A Survey of Graph Transformers: Architectures, Theories and Applications

arXiv:2502. 16533v3 Announce Type: replace-cross Abstract: Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing.

Introduction to Graph Machine Learning

Pre-training with Graph Transformers

The paper explores pre‑training methods for graph transformers applied to biochemistry. It finds that supervised pre‑training with computed property labels yields the best improvement on downstream tasks, and it emphasizes that limiting model capacity helps prevent overfitting.

Plain Transformers Can be Powerful Graph Learners

The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.

Introduction to Graph Neural Networks for Machine Learning Engineers

arXiv:2412. 19419v2 Announce Type: replace-cross Abstract: Graph neural networks are deep neural networks designed for graphs with attributes attached to nodes or edges.

Graph Neural Networks for the Graphical Bootstrap

arXiv:2607. 03109v1 Announce Type: cross Abstract: We study a graph classification problem involving over 20 million graphs, arising from high-order perturbative computations of correlators in planar $\mathcal{N}=4$ super-Yang--Mills, a model closely related to the theory of the strong nuclear force.