The paper introduces Topological Attention (Top‑A), a multi‑head attention mechanism that extends standard diagonal edge maps by allowing off‑diagonal, edge‑conditioned communication across attention heads. By isolating the transport primitive through quiver representations, the authors show that standard multi‑head attention only implements diagonal edge maps, whereas Top‑A learns additional cross‑head routes while preserving the original same‑head paths. Experiments on relational reasoning, heterogeneous graph learning, and algorithmic reasoning demonstrate that cross‑head transport is most beneficial when tasks require interaction‑dependent transformations, whereas heterophily alone does not provide a systematic advantage.
arXiv:2607. 10677v1 Announce Type: new Abstract: Self-attention is a ubiquitous primitive in modern sequence models, yet its operator-level geometry is only partially understood.
arXiv:2601. 21207v4 Announce Type: replace-cross Abstract: Combinatorial and topological structures, such as graphs, simplicial complexes, and cell complexes, form the foundation of geometric and topological deep learning (GDL and TDL) architectures.
arXiv:2608. 16097v1 Announce Type: new Abstract: Graph neural networks are commonly described through family-specific equations whose notation obscures shared computations and structural differences.
arXiv:2607. 01553v1 Announce Type: cross Abstract: Transformers have become general-purpose architectures, but their all-to-all self-attention is poorly matched to graph data, whose interactions are sparse, structured and multi-scale.
The paper introduces SheafDEQ, a subhomogeneous deep-equilibrium architecture that uses adaptive neural-sheaf propagation to allow richer, edge-dependent transformations in implicit graph neural networks while guaranteeing a unique equilibrium. The authors prove that SheafDEQ’s equilibrium is globally reachable from any positive initialization and remains contractive even with bounded communication staleness. Experiments demonstrate that SheafDEQ outperforms fixed-propagation implicit baselines on tasks such as Sums, MNIST Terrain, Coordinates, and community detection, especially as graph connectivity becomes increasingly heterophilic.
arXiv:2606. 10071v1 Announce Type: cross Abstract: We introduce Temporal Sheaf Neural Networks (TSNN), a temporal link prediction framework that equips each node with a time-varying orthogonal frame and compares node states only after explicit transport between local coordinate systems.
arXiv:2607. 28259v1 Announce Type: new Abstract: We introduce Topoformer, a lightweight and scalable framework for graph representation learning that encodes topological structure into attention-friendly sequences.
Message-passing Graph Neural Networks (GNNs) iteratively propagate and aggregate local neighborhood information followed by global readout to learn graph representations. However, their discriminative...
arXiv:2607. 19270v1 Announce Type: cross Abstract: The Traffic Assignment Problem is a fundamental but computationally expensive component of transportation planning.
arXiv:2607. 20484v1 Announce Type: new Abstract: Large Language Models (LLMs) are fundamentally limited by representation collapse, a bottleneck that severely degrades long-context performance.
arXiv:2609.06668v1 Announce Type: new Abstract: Multimodal graphs couple node attributes in different modalities, such as text and images, with relational structure, enabling topological structure an...
arXiv:2511. 11046v3 Announce Type: replace-cross Abstract: Graph neural networks (GNNs) have become an indispensable tool for analyzing relational data.