Symmetry governs the organization of physical structures, while symmetry breaking reveals their transformations. In geometric deep learning, symmetry imposes rank constraints on equivariant representa...
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.
arXiv:2608. 12010v1 Announce Type: new Abstract: Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields.
arXiv:2608.31045v1 Announce Type: new Abstract: Rotational symmetry is one of the most important structural principles in machine learning on 3D data. In applications ranging from physics and materia...
arXiv:2608. 05336v1 Announce Type: cross Abstract: Molecular representations are essential for the evaluation of molecular similarity and the development of structure-property relationships.
The paper investigates why Hessian spectra of trained deep learning models consistently show a bulk of eigenvalues near zero and a few isolated outliers. It proposes that these patterns arise from a hidden, highly symmetric reference configuration; when the model is perturbed (e.g., by architecture, data, or metric changes), the symmetry is broken, producing the observed cluster hierarchy. The authors analyze this mechanism in three‑layer ReLU networks and extend it to convolutional, graph, transformer models, the NTK, and related matrices such as layerwise Hessians and the Gauss‑Newton matrix.
arXiv:2608.28853v1 Announce Type: cross Abstract: Equivariant graph neural networks provide a principled way to model geometric systems, but efficient first-order architectures remain limited in how...
arXiv:2607. 03798v1 Announce Type: cross Abstract: Symmetry is everywhere in nature and society.
arXiv:2602. 10031v2 Announce Type: replace Abstract: Graph neural networks (GNNs) are commonly divided into message-passing neural networks (MPNNs) and spectral GNNs, reflecting two largely separate research traditions in machine learning and signal processing.
The paper investigates how parameter symmetries in overparameterized nonlinear neural networks affect representational geometry. It shows that these symmetries can be described by three basic feature transformations—addition, duplication, and scaling—allowing a closed‑form decomposition of representations into essential and auxiliary parts. The study further demonstrates that implementation‑level selection rules can resolve the resulting degeneracy, producing identifiable geometries that reflect feature contributions to the network’s function.
arXiv:2207. 03116v4 Announce Type: replace Abstract: We introduce a general method for learning representations that are equivariant to symmetries of data.
arXiv:2510. 14217v2 Announce Type: replace Abstract: The spectral properties of feature embeddings offer critical insights into model generalization and representation quality.