Hugging Face Trending Papers
Jun 24
Communicability-Inspired Positional Encoding (CIPE)
Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging.
Size Transferability of Graph Transformers with Convolutional Positional Encodings
Related stories
arXiv AI
Jul 28
Plain Transformers are Surprisingly Powerful Link Predictors
arXiv:2602. 01553v3 Announce Type: replace-cross Abstract: Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.
arXiv Machine Learning
Jun 25
Communicability-Inspired Positional Encoding (CIPE)
arXiv:2606. 25293v1 Announce Type: new Abstract: Positional encodings (PEs) are essential for Transformers.
arXiv Machine Learning
Aug 19
HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings
HyPE-GT introduces a framework that generates learnable hyperbolic positional encodings for Graph Transformers, enabling the capture of complex hierarchical relationships in graph-structured data. Unlike traditional Euclidean encodings, HyPE’s hyperbolic encodings can be selected to suit specific downstream tasks and help mitigate oversmoothing in deep Graph Neural Networks. Experiments on molecular benchmarks and large-scale Open Graph Benchmark datasets demonstrate improved performance, while additional tests on Coauthor and Copurchase networks confirm HyPE’s effectiveness in controlling oversmoothing.
arXiv Machine Learning
Aug 28
Plain Transformers Can be Powerful Graph Learners
The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.
arXiv AI
Jul 28
A Survey of Graph Transformers: Architectures, Theories and Applications
arXiv:2502. 16533v3 Announce Type: replace-cross Abstract: Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing.
arXiv Machine Learning
Jul 31
Train Small, Deploy Large: Zero-Shot GNN Transfer Through Geometric Renormalization
arXiv:2607. 27767v1 Announce Type: new Abstract: Graph neural networks (GNNs) can operate on large graphs but become infrastructure-sensitive at the scale of millions of nodes and typically require scalable training techniques for even larger graphs.
arXiv AI
Jun 18
Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers
arXiv:2503. 01805v3 Announce Type: replace-cross Abstract: Transformers have revolutionized the field of machine learning.
arXiv AI
Jun 26
Rotary Position Encodings for Graphs
arXiv:2509. 22259v4 Announce Type: replace-cross Abstract: We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transformers (ViTs), can be applied to graph-structured data.
arXiv AI
3d ago
Degraded but Not Entirely Ineffective: PE-Based Deformable Graph Neural Networks
The paper introduces a Position Encoding-Based Deformable Spatial Aggregation Module (PEBDSAM) that uses a deformable mechanism in position space to identify relevant nodes beyond first‑order neighbors, thereby addressing over‑smoothing, over‑compression, limited receptive fields, and noise from heterophilous graphs. Diagnostic experiments revealed that current offsets are ineffective, yet performance still improves, leading to a streamlined version called PEBSAM and a faster variant, PEBSAM‑Speed. The module is plug‑and‑play and can be integrated into GCN, GAT, GIN, and GraphSAGE, achieving strong results on both homophilous and heterophilous datasets.
arXiv AI
Jun 24
Graph Alignment for Benchmarking Graph Neural Networks and Learning Positional Encodings
arXiv:2505. 13087v2 Announce Type: replace-cross Abstract: We propose a novel benchmarking methodology for graph neural networks (GNNs) based on the graph alignment problem, a combinatorial optimization task that generalizes graph isomorphism by aligning two unlabeled graphs to maximize overlapping edges.
arXiv Machine Learning
Aug 20
GraphK: Variable-Size Graph Generation with Efficient Edge Construction
GraphK introduces an encoder‑sampler‑decoder framework that generates variable‑size graphs efficiently. It learns permutation‑invariant latent representations and samples new node embeddings via maximum likelihood, enabling both upscaling and downscaling of graph size. Edge construction uses KDTree‑based top‑k neighbor search in latent space, reducing computational cost while capturing graph properties.