arXiv AI
DTA-GT: Direction- and Topology-Aware Graph Transformer for Neural Network Representation Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 28
A Survey of Graph Transformers: Architectures, Theories and Applications
arXiv:2502. 16533v3 Announce Type: replace-cross Abstract: Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing.
arXiv AI
Sep 1
Decoupled Structure-Feature Alignment via Alternating Optimization for Graph Learning
arXiv:2607.11577v2 Announce Type: replace-cross Abstract: Conventional Graph Neural Networks (GNNs) couple feature transformation and neighborhood aggregation, which often renders them vulnerable to...
arXiv Machine Learning
Aug 28
Plain Transformers Can be Powerful Graph Learners
The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.
arXiv Machine Learning
Oct 23
Graph Memory Transformer (GMT)
The paper introduces the Graph Memory Transformer (GMT), a decoder‑only language model that replaces the traditional Feed‑Forward Network sublayer with an explicit learned memory graph. In GMT, token representations are routed through a bank of centroids linked by a directed transition matrix, using gravitational source routing and token‑conditioned target selection to produce gated displacements instead of retrieved values. The resulting model contains 82.2 M parameters, trains stably, and offers direct inspection of centroid usage and transition structure, though it slightly lags a larger GPT‑style baseline in validation loss and perplexity.
arXiv Machine Learning
Oct 8
Node-level Graph Neural Architecture Search Framework
The paper introduces N-GNAS, a Node-Level Graph Neural Architecture Search framework that selects distinct network architectures for different node subsets during feature updates. It incorporates a contrastive learning loss to better separate features across categories. Experiments on eight node and graph classification datasets show that N-GNAS outperforms existing GNAS methods and hand‑crafted GNNs, achieving 78.26% accuracy on CiteSeer.
Hugging Face Trending Papers
Oct 7
Node-level Graph Neural Architecture Search Framework
In recent years, Graph Neural Networks (GNNs) and architecture search frameworks have gained extensive application in non-Euclidean data processing, attributable to their superior capacity in managing...