FeatureFormer is a neural performance predictor that adds explicit node-wise encodings of FLOPs, parameter counts, and memory proxies to a gated graph attention architecture. It is designed to improve latency and energy prediction for neural networks on edge devices, addressing the limitation of existing GNN and transformer predictors that largely ignore node-level computational cost. The authors also introduce NNEQ, a large-scale energy consumption dataset, and show through extensive experiments that FeatureFormer achieves state‑of‑the‑art performance across both metrics, including challenging out‑of‑domain settings, while the encoding can broadly enhance existing predictors with negligible overhead.
By Matthew Grenier, William Hammer, Andrew Heuer, Nikhil Krishna, Yi Wang, Ramtin Zand
The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.
By Liheng Ma, Soumyasundar Pal, Yingxue Zhang, Philip H. S. Torr, Mark Coates
arXiv:2603. 08825v2 Announce Type: replace-cross Abstract: Discrete graph generation has emerged as a powerful paradigm for modeling graph-structured data, yet state of the art models often rely on Graph Transformers or higher order architectures.
By Jay Revolinsky, Harry Shomer, Jiliang Tang
arXiv:2607. 17272v1 Announce Type: new Abstract: Node representation learning has advanced rapidly, yet most existing methods rely on per-dataset training and hyperparameter tuning.
By Dooho Lee, Jaemin Yoo
arXiv:2607. 01553v1 Announce Type: cross Abstract: Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale.
By Leyan Li, Rennong Yang, Zhenxing Zhang, Liping Hu
The paper introduces gated graph attention and a learnable temperature to enhance standard graph attention networks. Gating filters out unreliable feature dimensions, while the temperature parameter adjusts the sharpness of attention coefficient distributions. Experiments on node classification and heterophilic benchmarks show that these modifications improve performance and robustness, with theoretical analysis supporting the observed gains.
By Zhongtian Ma, Hao Wu, Yexin Zhang, Qiaosheng Zhang, Chao Gao
arXiv:2602. 17071v3 Announce Type: replace-cross Abstract: Graph neural networks frequently encounter significant performance degradation when confronted with structural noise or non-homophilous topologies.
By Rong Fu, Muge Qi, Chunlei Meng, Shuo Yin, Kun Liu, Simon Fong
arXiv:2605.10247v2 Announce Type: replace
Abstract: Applying Large Language Models (LLMs) to graph-structured data usually involves multi-step pipelines in which textual node attributes are compresse...
By Dario Vajda
Deep Neural Sheaf Diffusion (DNSD) is a new graph neural network architecture that replaces the sheaf Laplacian with a sheaf adjacency operator to keep informative signals across many layers. It also uses normalization, odd nonlinearities, and gating, and replaces scalar attention with matrix-valued edge functions. Experiments show DNSD outperforms existing GNNs and Neural Sheaf Diffusion on both synthetic long‑range and real‑world graph benchmarks, achieving up to 30 percentage‑point gains in accuracy.
By R\'emi Bourgerie, \v{S}ar\=unas Girdzijauskas, Viktoria Fodor
arXiv:2604. 17324v2 Announce Type: replace-cross Abstract: Global self-attention drives modern graph transformers, yet the softmax at its core imposes a structural constraint rarely examined directly: every attention row is non-negative and sums to one, so each per-head output is a mass-conserving convex combination of value vectors.
By Yang Liu, Dongxin Guo, Tom Zheng, Siu Ming Yiu, Liam Ning, Jikun Wu
Graph-Dependent Pooling (GDPool) introduces a dynamic approach to hierarchical graph pooling, allowing each graph to determine its own pooling granularity rather than relying on fixed cluster numbers or ratios. The method employs self-attention to select graph-specific anchors and uses soft node assignment to build coarser graphs, addressing optimization conflicts identified through gradient alignment analysis. Experiments on metal‑organic framework and molecular graph datasets demonstrate that GDPool outperforms existing methods in predictive accuracy, modularity, and exhibits stable behavior across various hyperparameters.
By Bohan Yoon, Hyejin Jang
The paper introduces gated graph attention and a learnable temperature to enhance standard graph attention networks. Gated attention filters out unreliable feature dimensions, while the temperature parameter dynamically adjusts the sharpness of attention coefficient distributions. Experiments on both homogeneous and heterophilic heterogeneous benchmarks demonstrate consistent improvements over existing backbones, and controlled noise studies confirm the robustness of these mechanisms.
By Zhongtian Ma, Hao Wu, Yexin Zhang, Qiaosheng Zhang, Zhen Wang