Hugging Face Trending Papers
Jun 24
Communicability-Inspired Positional Encoding (CIPE)
Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging.
Communicability-Inspired Positional Encoding (CIPE)
arXiv:2606. 25293v1 Announce Type: new Abstract: Positional encodings (PEs) are essential for Transformers.
Related stories
arXiv Machine Learning
Sep 15
Size Transferability of Graph Transformers with Convolutional Positional Encodings
arXiv:2602.15239v3 Announce Type: replace Abstract: Transformers have achieved remarkable success across domains, motivating the rise of Graph Transformers (GTs) as attention-based architectures for...
arXiv Machine Learning
Aug 28
Plain Transformers Can be Powerful Graph Learners
The paper shows that a plain Transformer can serve as an effective graph learner by adding three lightweight modifications: simplified L₂ attention, adaptive RMS normalization, and an MLP-based positional encoding stem. These changes preserve token magnitude and enable the model to achieve high expressivity on graph benchmarks, outperforming more complex graph transformer variants. The results suggest that plain Transformers can act as a unified backbone for multimodal learning across language, vision, and graph domains.
arXiv AI
Jun 26
Rotary Position Encodings for Graphs
arXiv:2509. 22259v4 Announce Type: replace-cross Abstract: We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transformers (ViTs), can be applied to graph-structured data.
arXiv Machine Learning
Aug 19
HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings
HyPE-GT introduces a framework that generates learnable hyperbolic positional encodings for Graph Transformers, enabling the capture of complex hierarchical relationships in graph-structured data. Unlike traditional Euclidean encodings, HyPE’s hyperbolic encodings can be selected to suit specific downstream tasks and help mitigate oversmoothing in deep Graph Neural Networks. Experiments on molecular benchmarks and large-scale Open Graph Benchmark datasets demonstrate improved performance, while additional tests on Coauthor and Copurchase networks confirm HyPE’s effectiveness in controlling oversmoothing.
arXiv AI
Jun 17
Functional Equivalence in Attention: A Comprehensive Study with Applications to Linear Mode Connectivity
arXiv:2606. 17830v1 Announce Type: cross Abstract: Neural network parameter spaces are inherently non-injective, as distinct parameter configurations can realize identical functions through functional equivalence.
arXiv Machine Learning
Jul 21
A Weisfeiler-Leman Characterization of Global-Attention Graph Transformers for Mixed-Integer Linear Programs
arXiv:2607. 17570v1 Announce Type: new Abstract: Graph foundation models (GFMs) with global attention are increasingly used to represent mixed-integer linear programs (MILPs), aiming to capture structure beyond the locality of standard graph neural networks.
arXiv AI
Jul 28
A Survey of Graph Transformers: Architectures, Theories and Applications
arXiv:2502. 16533v3 Announce Type: replace-cross Abstract: Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing.
arXiv AI
Jul 3
X-LogSMask: Expand Transformer for Graph-Structured Data
arXiv:2607. 01553v1 Announce Type: cross Abstract: Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale.
arXiv Machine Learning
Jul 28
Unsupervised Graph Representation Learning with Complementary View Alignment
arXiv:2607. 24338v1 Announce Type: new Abstract: Unsupervised graph representation learning aims to derive meaningful node embeddings by capturing both structural and attribute information without relying on labeled data.
arXiv AI
Jun 4
A Unified Geometric Space for Topological Alignment Between Transformer-Based Models and Human Brain Networks
arXiv:2510. 24342v2 Announce Type: replace Abstract: Prior brain-AI alignment studies are typically constrained by specific inputs and tasks, limiting their ability to capture organizational properties across models with different modalities.
arXiv AI
Jun 24
Mind the Heads: Topological Representation Alignment for Multimodal LLMs
arXiv:2606. 23885v1 Announce Type: cross Abstract: Representation alignment has emerged as an effective approach to improve Multimodal Large Language Models (MLLMs) by regularizing their internal representations toward those of an external vision encoder.