arXiv:2502. 16533v3 Announce Type: replace-cross Abstract: Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing.
arXiv:2607.11577v2 Announce Type: replace-cross Abstract: Conventional Graph Neural Networks (GNNs) couple feature transformation and neighborhood aggregation, which often renders them vulnerable to...
The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.
The paper introduces the Graph Memory Transformer (GMT), a decoder‑only language model that replaces the traditional Feed‑Forward Network sublayer with an explicit learned memory graph. In GMT, token representations are routed through a bank of centroids linked by a directed transition matrix, using gravitational source routing and token‑conditioned target selection to produce gated displacements instead of retrieved values. The resulting model contains 82.2 M parameters, trains stably, and offers direct inspection of centroid usage and transition structure, though it slightly lags a larger GPT‑style baseline in validation loss and perplexity.
The paper introduces N-GNAS, a Node-Level Graph Neural Architecture Search framework that selects distinct network architectures for different node subsets during feature updates. It incorporates a contrastive learning loss to better separate features across categories. Experiments on eight node and graph classification datasets show that N-GNAS outperforms existing GNAS methods and hand‑crafted GNNs, achieving 78.26% accuracy on CiteSeer.
In recent years, Graph Neural Networks (GNNs) and architecture search frameworks have gained extensive application in non-Euclidean data processing, attributable to their superior capacity in managing...
FeatureFormer is a neural performance predictor that adds explicit node-wise encodings of FLOPs, parameter counts, and memory proxies to a gated graph attention architecture. It is designed to improve latency and energy prediction for neural networks on edge devices, addressing the limitation of existing GNN and transformer predictors that largely ignore node-level computational cost. The authors also introduce NNEQ, a large-scale energy consumption dataset, and show through extensive experiments that FeatureFormer achieves state‑of‑the‑art performance across both metrics, including challenging out‑of‑domain settings, while the encoding can broadly enhance existing predictors with negligible overhead.
arXiv:2607. 11577v1 Announce Type: cross Abstract: We introduce a constrained two-view framework for node prediction that aligns structure-conditioned GNN embeddings with a structure-free feature prior learned by an anchor model.
The paper introduces RGC‑Net, a Reservoir‑Based Graph Convolutional Network that combines fixed‑random reservoir dynamics with a structured convolutional framework for graph learning. It addresses limitations of existing reservoir‑based GNNs by adding a leaky integrator for better feature retention and a robust, adaptable architecture for graph classification and generation. Experiments demonstrate state‑of‑the‑art performance on classification and generative tasks, including dynamic brain connectivity, with faster convergence and reduced over‑smoothing.
The paper investigates how self‑supervised graph representations can complement supervised ones for out‑of‑distribution (OOD) node classification. Two backbone‑agnostic frameworks are introduced: Co‑Train, which jointly learns and integrates supervised and SSL representations during training, and Dual‑Space Retrieval, which performs non‑parametric prediction in both representation spaces and fuses the results at inference. Experiments on four graph benchmarks with various GNN backbones and SSL objectives (DGI and GRACE) show that Co‑Train consistently outperforms strong supervised OOD baselines, while Dual‑Space Retrieval offers competitive, flexible performance, demonstrating that SSL provides complementary signals that improve OOD node classification.
arXiv:2602. 01553v3 Announce Type: replace-cross Abstract: Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.
Out-of-distribution (OOD) generalization remains challenging for graph neural networks (GNNs), as graph distributions can vary substantially across time and domains. Supervised and self-supervised gra...