Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.
arXiv:2608. 12010v1 Announce Type: new Abstract: Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields.
CrossGMN introduces a graph metanetwork that processes a trained source network and an initialized target network simultaneously, enabling equivariant cross‑architecture weight‑space transformations. By preserving symmetry through cross‑network message passing, CrossGMN can refine target network initializations while remaining invariant to source permutations and equivariant to target permutations. Experiments demonstrate that CrossGMN accelerates knowledge distillation, transfers across datasets without retraining, and unifies compression from diverse source architectures into a common target architecture.
arXiv:2606. 04754v1 Announce Type: new Abstract: Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformations that leave the realized function unchanged.
arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.
The paper investigates how a vanilla Transformer learns symmetries from finite data augmentation on point cloud datasets. It finds an ordering of learnability: non-angle-preserving symmetries are easiest, followed by angle-preserving symmetries, and finally base angle-preserving subgroups such as translation, rotation, and scale. The study also examines the Transformer's extrapolation behavior, performs a structural analysis of trained models to uncover interpretable mechanisms for invariance, and extends these findings to equivariant functions, suggesting that the identified mechanisms can serve as building blocks for learned equivariance.
arXiv:2606. 10913v1 Announce Type: new Abstract: We explore whether intrinsic symmetries of the training data lead to conserved quantities during gradient-flow training of neural networks.
The paper investigates how parameter symmetries in overparameterized nonlinear neural networks affect representational geometry. It shows that these symmetries can be described by three basic feature transformations—addition, duplication, and scaling—allowing a closed‑form decomposition of representations into essential and auxiliary parts. The study further demonstrates that implementation‑level selection rules can resolve the resulting degeneracy, producing identifiable geometries that reflect feature contributions to the network’s function.
arXiv:2610.17269v1 Announce Type: cross Abstract: Learning symmetries through data augmentation offers an alternative to architectural equivariance, but does not guarantee that a network respects tho...
arXiv:2410. 06665v4 Announce Type: replace-cross Abstract: This paper explores the characterization of equivariant linear layers for representations of permutations and related groups.
arXiv:2605. 18106v3 Announce Type: replace-cross Abstract: A striking geometric disparity has long persisted in the practice of deep learning.
arXiv:2509. 11218v2 Announce Type: replace-cross Abstract: Spatial transformations such as rotation and scale obscure the morphological cues needed for accurate image classification.