arXiv Machine Learning
Jul 7
A Complete Symmetry Classification of Shallow ReLU Networks
arXiv:2604. 14037v2 Announce Type: replace Abstract: Parameter space is not function space for neural network architectures.
Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability
arXiv:2606. 04754v1 Announce Type: new Abstract: Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformations that leave the realized function unchanged.
Related stories
arXiv Machine Learning
Jun 10
Conservation Laws from Data Symmetry in Neural Networks
arXiv:2606. 10913v1 Announce Type: new Abstract: We explore whether intrinsic symmetries of the training data lead to conserved quantities during gradient-flow training of neural networks.
Hugging Face Trending Papers
Aug 12
Reducing Symmetry Increase in Equivariant Neural Networks
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries.
arXiv Machine Learning
6d ago
Reducing Symmetry Increase in Equivariant Neural Networks
arXiv:2608. 12010v1 Announce Type: new Abstract: Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields.
arXiv Machine Learning
Jul 10
Explaining Near-Zero Hessian Eigenvalues Through Approximate Symmetries in Neural Networks
arXiv:2607. 07845v1 Announce Type: new Abstract: The Hessian of the training loss governs the local geometry of the loss landscape, yet despite existing explanations for its largest eigenvalues, the origin of the vast multitude of vanishingly small eigenvalues remains elusive.
arXiv Machine Learning
Aug 10
Equivariant Sparse Autoencoders: Mechanistic Interpretability of Neural Networks on Symmetric Data
arXiv:2511. 09432v2 Announce Type: replace Abstract: Machine learning (ML) models achieve remarkable performance but remain hard to interpret due to their scale and complexity.
arXiv Machine Learning
Jun 18
InTrain: Intrinsic Trainability for Zero-Cost Neural Architecture Search
arXiv:2606. 18676v1 Announce Type: new Abstract: Training-free neural architecture search promises efficient discovery of high-performance networks without costly training.
arXiv Machine Learning
Jul 8
Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity
arXiv:2607. 05546v1 Announce Type: cross Abstract: We develop a unified function space theory of deep fully connected neural networks.
arXiv Machine Learning
Jul 9
Network Dynamics-Based Framework for Understanding Deep Neural Networks
arXiv:2501. 02436v5 Announce Type: replace Abstract: Advancements in artificial intelligence call for a deeper understanding of the fundamental mechanisms underlying deep learning.
arXiv Machine Learning
Jun 5
Gradient Descent with Large Step Size Restores Symmetry in Deep Linear Networks with Multi-Pathway
arXiv:2606. 05219v1 Announce Type: new Abstract: Recent analyses of multi-pathway Deep Linear Networks use Gradient Flow to predict a "winner-takes-all" specialization in which path symmetry breaks and each feature concentrates in a single pathway.
arXiv AI
Jul 14
Learning in Curved Weight Space:Exponential-Linear Weight Reparameterization for Improved Optimization
arXiv:2607. 09967v1 Announce Type: cross Abstract: Many neural networks operations have a multiplicative nature rather than additive: halving or doubling a norm are analogous relatively but require unequal optimization distances when taking linear steps.
arXiv Machine Learning
Jul 30
Automorphism-Induced Non-Canonicity in Top-k Explanations of Graph Neural Networks
arXiv:2607. 26344v1 Announce Type: new Abstract: A gradient-based GNN explainer given a molecule with two chemically equivalent nitro groups assigns them attribution scores that are equal to the last bit.