Artificial neural networks generate local symmetries called fibrations and coverings during learning, and these covering symmetries are stable attractors of stochastic gradient descent. The study shows that such symmetries appear across diverse architectures—multilayer, convolutional, recurrent, and transformer networks—and can be exploited for drastic model compression, reducing networks to 17% of their original size without performance loss. Controlled breaking of covering symmetry further improves continual learning, achieving state‑of‑the‑art results.
arXiv:2606. 04754v1 Announce Type: new Abstract: Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformations that leave the realized function unchanged.
The paper introduces a general theoretical framework for fibrations on graphs labeled by a commutative monoid, extending the classic theory of graph fibrations to weighted and algebraically labeled graphs. It also accommodates approximate fibrations and demonstrates how this framework can be used to compress arbitrary neural networks, including CNNs, providing a solid theoretical basis for recent findings on fibration symmetries in geometric deep learning.
The paper investigates how parameter symmetries in overparameterized nonlinear neural networks affect representational geometry. It shows that these symmetries can be described by three basic feature transformations—addition, duplication, and scaling—allowing a closed‑form decomposition of representations into essential and auxiliary parts. The study further demonstrates that implementation‑level selection rules can resolve the resulting degeneracy, producing identifiable geometries that reflect feature contributions to the network’s function.
arXiv:2606. 10913v1 Announce Type: new Abstract: We explore whether intrinsic symmetries of the training data lead to conserved quantities during gradient-flow training of neural networks.
arXiv:2610.17424v1 Announce Type: cross Abstract: Respecting a-priori known symmetries of underlying data is a key principle in designing efficient neural architectures. Convolutional neural networks...
The paper introduces a physics-inspired neural framework that combines graph neural networks with statistical‑mechanics principles to solve large‑scale graph coloring problems. By using a planting‑based supervised signal, symmetry‑breaking regularization, and iterative noise‑annealed neural dynamics, the model navigates complex solution landscapes and scales quadratically with graph size. It generalizes from small training graphs to instances up to 20 times larger, demonstrating that neural architectures can learn scalable algorithmic strategies effective in hard connectivity regions.
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.
arXiv:2501. 02436v5 Announce Type: replace Abstract: Advancements in artificial intelligence call for a deeper understanding of the fundamental mechanisms underlying deep learning.
arXiv:2507. 10005v2 Announce Type: replace Abstract: In recent years, graph-based machine learning techniques, such as reinforcement learning and graph neural networks, have garnered significant attention.
arXiv:2608. 12010v1 Announce Type: new Abstract: Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields.
arXiv:2606. 05046v1 Announce Type: new Abstract: We introduce Graph Cascades, a mesoscopic rewiring strategy for Graph Neural Networks (GNNs) and Graph Transformers (GTs) that captures intermediate-scale graph structure beyond purely local edges or fully global attention.
arXiv:2507. 05164v2 Announce Type: replace-cross Abstract: In this chapter, we utilize dynamical systems to analyze several aspects of machine learning algorithms.