arXiv AI
On Making $\mathrm{SE}(2)$-Invariant Networks Optimal
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 12
Reducing Symmetry Increase in Equivariant Neural Networks
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.
arXiv Machine Learning
Aug 13
Reducing Symmetry Increase in Equivariant Neural Networks
arXiv:2608. 12010v1 Announce Type: new Abstract: Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields.
arXiv Machine Learning
Oct 3
CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations
CrossGMN introduces a graph metanetwork that processes a trained source network and an initialized target network simultaneously, enabling equivariant cross‑architecture weight‑space transformations. By preserving symmetry through cross‑network message passing, CrossGMN can refine target network initializations while remaining invariant to source permutations and equivariant to target permutations. Experiments demonstrate that CrossGMN accelerates knowledge distillation, transfers across datasets without retraining, and unifies compression from diverse source architectures into a common target architecture.
arXiv Machine Learning
Jun 4
Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability
arXiv:2606. 04754v1 Announce Type: new Abstract: Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformations that leave the realized function unchanged.
arXiv Machine Learning
Aug 10
Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data
arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.
arXiv Machine Learning
Oct 8
How Do Transformers Learn to Represent Symmetries?
The paper investigates how a vanilla Transformer learns symmetries from finite data augmentation on point cloud datasets. It finds an ordering of learnability: non-angle-preserving symmetries are easiest, followed by angle-preserving symmetries, and finally base angle-preserving subgroups such as translation, rotation, and scale. The study also examines the Transformer's extrapolation behavior, performs a structural analysis of trained models to uncover interpretable mechanisms for invariance, and extends these findings to equivariant functions, suggesting that the identified mechanisms can serve as building blocks for learned equivariance.