Hugging Face Trending Papers
Jun 24
Communicability-Inspired Positional Encoding (CIPE)
Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging.
Rotary Position Encodings for Graphs
arXiv:2509. 22259v4 Announce Type: replace-cross Abstract: We study the extent to which rotary position encodings (RoPE), a recent transformer position encoding algorithm broadly adopted in large language models (LLMs) and vision transformers (ViTs), can be applied to graph-structured data.
Related stories
arXiv Machine Learning
Jun 25
Communicability-Inspired Positional Encoding (CIPE)
arXiv:2606. 25293v1 Announce Type: new Abstract: Positional encodings (PEs) are essential for Transformers.
arXiv Machine Learning
22h ago
HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings
arXiv:2312. 06576v2 Announce Type: replace Abstract: Graph Transformers (GTs) facilitate the comprehension of complex relationships on graph-structured data by leveraging self-attention of the possible pairs of nodes.
arXiv AI
Jul 28
Plain Transformers are Surprisingly Powerful Link Predictors
arXiv:2602. 01553v3 Announce Type: replace-cross Abstract: Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.
arXiv AI
Jul 3
X-LogSMask: Expand Transformer for Graph-Structured Data
arXiv:2607. 01553v1 Announce Type: cross Abstract: Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale.
arXiv AI
Jun 17
Functional Equivalence in Attention: A Comprehensive Study with Applications to Linear Mode Connectivity
arXiv:2606. 17830v1 Announce Type: cross Abstract: Neural network parameter spaces are inherently non-injective, as distinct parameter configurations can realize identical functions through functional equivalence.
arXiv Machine Learning
Jun 24
Selective Rotary Position Embedding
arXiv:2511. 17388v3 Announce Type: replace-cross Abstract: Position information is essential for language modeling.
arXiv Machine Learning
Jul 14
LeRoPE: Learnable RoPE Frequencies Improve Language Modeling
arXiv:2607. 10134v1 Announce Type: new Abstract: Rotary Positional Encodings (RoPE) are currently the most popular positional encodings used in modern language models.
arXiv Machine Learning
Jun 18
Formalizing and Mitigating Structural Distortion in LLM Attention for Graph Reasoning
arXiv:2606. 15633v2 Announce Type: replace Abstract: Large Language Models (LLMs) have shown promise for reasoning over Text-Attributed Graphs (TAGs).
arXiv AI
Jul 15
Higher Embedding Dimension Creates a Stronger World Model for a Simple Sorting Task
arXiv:2510. 18315v2 Announce Type: replace-cross Abstract: We investigate how embedding dimension affects the emergence of an internal "world model" in a transformer trained with reinforcement learning to perform bubble-sort-style adjacent swaps.
arXiv Machine Learning
Jun 3
When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models
arXiv:2606. 03712v1 Announce Type: new Abstract: Graph Language Models (GLMs) have become a promising direction for adapting Large Language Models (LLMs) to graph learning tasks.
arXiv AI
Jun 11
RoVE: Rotary Value Embeddings Attention for Relative Position-dependent Value Pathways
arXiv:2606. 11275v1 Announce Type: cross Abstract: Rotary Position Embeddings (RoPE) make attention scores position-relative but leave the value pathway position-blind: the message sent by a value token is the same regardless of its distance from the query.