Hugging Face Trending Papers
Jun 24
Communicability-Inspired Positional Encoding (CIPE)
Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging.
Enhancing Transformer-based Routing by Encoding Distance via Relative Positional Encoding
arXiv:2607. 18909v1 Announce Type: new Abstract: This paper explores Relative Positional Encoding (RPE) as an additive bias in Transformer architectures to solve the Team Orienteering Problem.
Related stories
arXiv Machine Learning
Jun 25
Communicability-Inspired Positional Encoding (CIPE)
arXiv:2606. 25293v1 Announce Type: new Abstract: Positional encodings (PEs) are essential for Transformers.
arXiv AI
Jun 26
Rotary Position Encodings for Graphs
arXiv:2509. 22259v4 Announce Type: replace-cross Abstract: We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transformers (ViTs), can be applied to graph-structured data.
arXiv AI
Jul 15
Higher Embedding Dimension Creates a Stronger World Model for a Simple Sorting Task
arXiv:2510. 18315v2 Announce Type: replace-cross Abstract: We investigate how embedding dimension affects the emergence of an internal "world model" in a transformer trained with reinforcement learning to perform bubble-sort-style adjacent swaps.
arXiv Machine Learning
Aug 10
Graph Machine: Exploring Edge Mechanisms as an Inductive Bias
arXiv:2608. 06834v1 Announce Type: new Abstract: Transformers provide a powerful architecture for global content-based matching, but reasoning problems may benefit from a stronger inductive bias toward iterative traversal of latent relations.
arXiv AI
Jun 4
Do Transformers Need Three Projections? Systematic Study of QKV Variants
arXiv:2606. 04032v1 Announce Type: cross Abstract: Transformers have become the standard solution for various AI tasks, with the query, key, and value (QKV) attention formulation playing a central role.
arXiv Machine Learning
23h ago
HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings
arXiv:2312. 06576v2 Announce Type: replace Abstract: Graph Transformers (GTs) facilitate the comprehension of complex relationships on graph-structured data by leveraging self-attention of the possible pairs of nodes.
arXiv AI
Jun 29
Symmetry-Aware Transformer Training for Automated Planning
arXiv:2508. 07743v2 Announce Type: replace Abstract: While transformers excel in many settings, their application in the field of automated planning is limited.
arXiv AI
Jul 28
Plain Transformers are Surprisingly Powerful Link Predictors
arXiv:2602. 01553v3 Announce Type: replace-cross Abstract: Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.
arXiv AI
1d ago
Geometric Self-Supervised Pre-training for Neural Combinatorial Optimization
arXiv:2608. 00270v2 Announce Type: replace Abstract: Neural Combinatorial Optimization (NCO) techniques have emerged as a highly efficient alternative to traditional exact algorithms for solving routing problems such as the Traveling Salesman Problem (TSP).
arXiv Machine Learning
Jun 18
Complementary Attention Head Pruning for Efficient Transformers
arXiv:2606. 19150v1 Announce Type: new Abstract: The remarkable success of Transformer-based models in natural language processing stems from architectural scaling, which leads to a large number of parameters and hinders deployment in resource-constrained environments.
arXiv AI
Jul 22
On the Effectiveness of Pretraining for Graph Combinatorial Optimization
arXiv:2607. 19072v1 Announce Type: new Abstract: This paper introduces a self-supervised pretraining framework for graph combinatorial optimization specifically designed to address the nature of routing problems like the Traveling Salesman Problem.