The paper introduces gated graph attention and a learnable temperature to enhance standard graph attention networks. Gating filters out unreliable feature dimensions, while the temperature parameter adjusts the sharpness of attention coefficient distributions. Experiments on node classification and heterophilic benchmarks show that these modifications improve performance and robustness, with theoretical analysis supporting the observed gains.
By Zhongtian Ma, Hao Wu, Yexin Zhang, Qiaosheng Zhang, Chao Gao
arXiv:2610.28618v1 Announce Type: new
Abstract: Graph generative models commonly use graph-specific Transformer architectures to jointly update node and edge features. However, these architectures do...
By Vincent Jung, Alba Carballo-Castro, Yiming Qin, Lonneke van der Plas, Pascal Frossard
arXiv:2609.14663v1 Announce Type: new
Abstract: Deep neural networks are highly over-parameterized, and different parameter values represent the same predictive function. This makes their effective c...
By Vishnu Varadarajan, Mihir More, Aritra Das, Debayan Gupta
arXiv:2508. 17821v3 Announce Type: replace-cross Abstract: This paper investigates the limitations of the normalization in attention mechanisms.
By Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova, Radu State
arXiv:2604. 07492v2 Announce Type: replace-cross Abstract: Message Passing Neural Networks have recently become the most popular approach to graph machine learning tasks; however, their receptive field is limited by the number of message passing layers.
By Oleg Platonov, Liudmila Prokhorenkova
FeatureFormer is a neural performance predictor that adds explicit node-wise encodings of FLOPs, parameter counts, and memory proxies to a gated graph attention architecture. It is designed to improve latency and energy prediction for neural networks on edge devices, addressing the limitation of existing GNN and transformer predictors that largely ignore node-level computational cost. The authors also introduce NNEQ, a large-scale energy consumption dataset, and show through extensive experiments that FeatureFormer achieves state‑of‑the‑art performance across both metrics, including challenging out‑of‑domain settings, while the encoding can broadly enhance existing predictors with negligible overhead.
By Matthew Grenier, William Hammer, Andrew Heuer, Nikhil Krishna, Yi Wang, Ramtin Zand