Google AI Blog

Exphormer: Scaling transformers for graph-structured data

Posted by Ameya Velingker, Research Scientist, Google Research, and Balaji Venkatachalam, Software Engineer, Google Graphs , in which objects and their relations are represented as nodes (or vertices) and edges (or links) between pairs of nodes, are ubiquitous in computing and machine learning (ML). For example, social networks, road networks, and molecular structure and interactions are all domains in which underlying datasets have a natural graph structure.

Google AI Blog
Feb 6, 2024

Graph neural networks in TensorFlow

Posted by Dustin Zelle, Software Engineer, Google Research, and Arno Eigenwillig, Software Engineer, CoreML Objects and their relationships are ubiquitous in the world around us, and relationships can be as important to understanding an object as its own attributes viewed in isolation — take for example transportation networks, production networks, knowledge graphs, or social networks. Discrete mathematics and computer science have a long history of formalizing such networks as graphs , consisting of nodes connected by edges in various irregular ways.

By Google AI
arXiv AI
Jul 28

A Survey of Graph Transformers: Architectures, Theories and Applications

arXiv:2502. 16533v3 Announce Type: replace-cross Abstract: Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing.

By Chaohao Yuan, Kangfei Zhao, Ercan Engin Kuruoglu, Liang Wang, Tingyang Xu, Wenbing Huang, Deli Zhao, Hong Cheng, Yu Rong
arXiv AI
Sep 18

SCGFM-ART: Amortized Relational Transport for Structure-Centric Graph Foundation Models

SCGFM-ART is a structure‑centric graph foundation model that aligns arbitrary graphs onto a shared relational atlas using Amortized Relational Transport (ART). The atlas, defined by a finite set of relational landmarks, provides a universal coordinate system, while ART predicts end‑to‑end graph‑to‑base transport plans, eliminating costly runtime Gromov‑Wasserstein optimizations. The framework decomposes graphs into global relational response coordinates and local node‑to‑role structural correspondences, enabling unified representations that resolve structural and semantic heterogeneity across diverse graph domains.

By Xiaodong He, Xincheng Wang, Zhao Kang
arXiv Machine Learning
Aug 28

Plain Transformers Can be Powerful Graph Learners

The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.

By Liheng Ma, Soumyasundar Pal, Yingxue Zhang, Philip H. S. Torr, Mark Coates