arXiv Machine Learning
No Equivariant Architecture Covers All Equivariant Attention
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 29
A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of $\mathrm{O}(2)$
arXiv:2606. 27864v1 Announce Type: cross Abstract: Vision transformers have become a dominant architecture for visual recognition.
arXiv Machine Learning
Sep 2
Scaled Idempotence in Transformer Attention: Paired OV Geometry and Shared-Value Algebras
arXiv:2609.01129v1 Announce Type: new Abstract: We identify a recurrent algebraic regularity in Transformer attention: a sparse subset of effective OV operators $T=OV^\top$ nearly closes under compos...
arXiv AI
Jun 3
Exact equivariance, kept through training, buys zero-shot generalisation across the symmetry group
arXiv:2606. 03003v1 Announce Type: cross Abstract: A latent world model built from an equivariant encoder $E$ and an equivariant predictor $f$ inherits a provable symmetry of its training loss: when the world's dynamics genuinely carries a group $G$ acting on latents by an orthogonal representation $\rho(g)$, the one-step prediction relMSE is exactly invariant across the whole group, so fitting the dynamics on a restricted slice of orientations mathematically determines it on the entire orbit (j\v{u} y\=i f\v{a}n s\=an).
arXiv Machine Learning
Aug 4
Polyatomic Complexes: A topologically-informed learning representation for atomistic systems
arXiv:2409. 15600v3 Announce Type: replace Abstract: A representation of a molecule or material should be invariant to the symmetries of physics, unique, continuous, efficient and general.
arXiv AI
Aug 3
Revisiting Multi-Permutation Equivariance through the Lens of Irreducible Representations
arXiv:2410. 06665v4 Announce Type: replace-cross Abstract: This paper explores the characterization of equivariant linear layers for representations of permutations and related groups.
arXiv Machine Learning
Jun 19
The Token Is a Group Element: On Lie-Algebra Attention over Matrix Lie Groups
arXiv:2606. 20547v1 Announce Type: new Abstract: We place the attention token on the group: a token is an element $g_i$ of a matrix Lie group $G$ -- a bare transformation, with no feature payload and no external action $\rho(g)$ carrying it.