arXiv Machine Learning

Pre-training with Graph Transformers

The paper explores pre‑training methods for graph transformers applied to biochemistry. It finds that supervised pre‑training with computed property labels yields the best improvement on downstream tasks, and it emphasizes that limiting model capacity helps prevent overfitting.

arXiv AI
Oct 27

Graph-dependent pooling: adaptive granularity in hierarchical graph neural networks

Graph-Dependent Pooling (GDPool) introduces a dynamic approach to hierarchical graph pooling, allowing each graph to determine its own pooling granularity rather than relying on fixed cluster numbers or ratios. The method employs self-attention to select graph-specific anchors and uses soft node assignment to build coarser graphs, addressing optimization conflicts identified through gradient alignment analysis. Experiments on metal‑organic framework and molecular graph datasets demonstrate that GDPool outperforms existing methods in predictive accuracy, modularity, and exhibits stable behavior across various hyperparameters.

By Bohan Yoon, Hyejin Jang
arXiv AI
Jun 17

Handling Feature Heterogeneity with Learnable Graph Patches

arXiv:2606. 17667v1 Announce Type: cross Abstract: In recent years, the rapid development of foundation models and graph pre-training technologies has spurred increasing interest in constructing a universal pre-trained graph model or Graph Foundation Model (GFM).

By Yifei Sun, Yang Yang, Xiao Feng, Zijun Wang, Haoyang Zhong, Chunping Wang, Lei Chen
arXiv Machine Learning
Jun 4

Graph Set Transformer

arXiv:2606. 05116v1 Announce Type: new Abstract: We introduce the Graph Set Transformer (GST), a neural network architecture for learning on sets of graphs, designed for tasks in which per-element predictions depend on set-wide context as well as local structure.

By Jose E. Escrig Molina, Baoquan Chen, Daniel Probst
arXiv Machine Learning
Aug 19

HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings

HyPE-GT introduces a framework that generates learnable hyperbolic positional encodings for Graph Transformers, enabling the capture of complex hierarchical relationships in graph-structured data. Unlike traditional Euclidean encodings, HyPE’s hyperbolic encodings can be selected to suit specific downstream tasks and help mitigate oversmoothing in deep Graph Neural Networks. Experiments on molecular benchmarks and large-scale Open Graph Benchmark datasets demonstrate improved performance, while additional tests on Coauthor and Copurchase networks confirm HyPE’s effectiveness in controlling oversmoothing.

By Kushal Bose, Swagatam Das