arXiv:2607. 18909v1 Announce Type: new Abstract: This paper explores Relative Positional Encoding (RPE) as an additive bias in Transformer architectures to solve the Team Orienteering Problem.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging.
arXiv:2606. 25293v1 Announce Type: new Abstract: Positional encodings (PEs) are essential for Transformers.
The paper investigates Graph Transformers (GTs) that use Graph Neural Network (GNN)-based positional encodings. By analyzing GTs through manifold limit models, the authors establish a theoretical link between GTs and Manifold Neural Networks (MNNs), showing that GTs inherit transferability guarantees from their positional encodings. Experiments on standard graph benchmarks confirm that GTs scale comparably to GNNs, and a real‑world shortest‑path estimation task demonstrates their practical efficiency on large terrains.
arXiv:2602.15239v3 Announce Type: replace Abstract: Transformers have achieved remarkable success across domains, motivating the rise of Graph Transformers (GTs) as attention-based architectures for...
arXiv:2509. 22259v4 Announce Type: replace-cross Abstract: We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transformers (ViTs), can be applied to graph-structured data.
arXiv:2510. 18315v2 Announce Type: replace-cross Abstract: We investigate how embedding dimension affects the emergence of an internal "world model" in a transformer trained with reinforcement learning to perform bubble-sort-style adjacent swaps.