← Back to all news
arXiv Machine Learning By Osvaldo M Velarde, Lucas C Parra, Alireza Hashemi, Hernan A Makse

Emergence of Fibrations, Compression, and Symmetry Breaking in Artificial Neural Networks

Read the original on arXiv Machine Learning →

Artificial neural networks generate local symmetries called fibrations and coverings during learning, and these covering symmetries are stable attractors of stochastic gradient descent. The study shows that such symmetries appear across diverse architectures—multilayer, convolutional, recurrent, and transformer networks—and can be exploited for drastic model compression, reducing networks to 17% of their original size without performance loss. Controlled breaking of covering symmetry further improves continual learning, achieving state‑of‑the‑art results.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 27

M-Fibration Theory with Applications to Neural Network Compression

The paper introduces a general theoretical framework for fibrations on graphs labeled by a commutative monoid, extending the classic theory of graph fibrations to weighted and algebraically labeled graphs. It also accommodates approximate fibrations and demonstrates how this framework can be used to compress arbitrary neural networks, including CNNs, providing a solid theoretical basis for recent findings on fibration symmetries in geometric deep learning.

By Paolo Boldi
More like this →
arXiv Machine Learning
Oct 2

Parameter symmetries determine representational geometry in overparameterized nonlinear networks

The paper investigates how parameter symmetries in overparameterized nonlinear neural networks affect representational geometry. It shows that these symmetries can be described by three basic feature transformations—addition, duplication, and scaling—allowing a closed‑form decomposition of representations into essential and auxiliary parts. The study further demonstrates that implementation‑level selection rules can resolve the resulting degeneracy, producing identifiable geometries that reflect feature contributions to the network’s function.

By Marvin Theiss, Lukas Braun, Andrew M. Saxe, Erin Grant
More like this →
arXiv Machine Learning
Oct 21

Physics Inspired Neural Networks Enable Efficient Graph Coloring

The paper introduces a physics-inspired neural framework that combines graph neural networks with statistical‑mechanics principles to solve large‑scale graph coloring problems. By using a planting‑based supervised signal, symmetry‑breaking regularization, and iterative noise‑annealed neural dynamics, the model navigates complex solution landscapes and scales quadratically with graph size. It generalizes from small training graphs to instances up to 20 times larger, demonstrating that neural architectures can learn scalable algorithmic strategies effective in hard connectivity regions.

By Lorenzo Colantonio (Department of Physics, Sapienza University of Rome), Andrea Cacioppo (Department of Physics, Sapienza University of Rome), Federico Scarpati (Department of Physics, Sapienza University of Rome), Maria Chiara Angelini (Department of Physics, Sapienza University of Rome), Federico Ricci-Tersenghi (Department of Physics, Sapienza University of Rome), Stefano Giagu (Department of Physics, Sapienza University of Rome)
More like this →