arXiv:2502. 16533v3 Announce Type: replace-cross Abstract: Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing.
The paper explores pre‑training methods for graph transformers applied to biochemistry. It finds that supervised pre‑training with computed property labels yields the best improvement on downstream tasks, and it emphasizes that limiting model capacity helps prevent overfitting.
The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.
arXiv:2412. 19419v2 Announce Type: replace-cross Abstract: Graph neural networks are deep neural networks designed for graphs with attributes attached to nodes or edges.
arXiv:2607. 03109v1 Announce Type: cross Abstract: We study a graph classification problem involving over 20 million graphs, arising from high-order perturbative computations of correlators in planar $\mathcal{N}=4$ super-Yang--Mills, a model closely related to the theory of the strong nuclear force.
arXiv:2509. 22259v4 Announce Type: replace-cross Abstract: We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transformers (ViTs), can be applied to graph-structured data.
arXiv:2602. 01553v3 Announce Type: replace-cross Abstract: Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.
arXiv:2602.15239v3 Announce Type: replace Abstract: Transformers have achieved remarkable success across domains, motivating the rise of Graph Transformers (GTs) as attention-based architectures for...
The article reviews the problem of learning graph structures from data, noting that research has traditionally split into two paths: inferring the topology of a single graph from observations on it, and learning a generative distribution from multiple observed graphs to sample new ones. It proposes a unified framework that treats both as inverse problems of a common graph generation process, reviews key methods, and discusses their interrelations, strengths, and limitations. The review highlights opportunities for cross‑paradigm integration and outlines future research directions.
GraphIFE addresses the class imbalance problem in graph-structured data by tackling a quality inconsistency issue in synthesized nodes. The framework uses graph invariant learning to strengthen embedding space representations and identify invariant features, leading to improved performance on minority classes. Experiments show that GraphIFE consistently outperforms various baselines across multiple datasets.
Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging.