arXiv Machine Learning
Jul 7
A Complete Symmetry Classification of Shallow ReLU Networks
arXiv:2604. 14037v2 Announce Type: replace Abstract: Parameter space is not function space for neural network architectures.
Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks
arXiv:2607. 08561v1 Announce Type: new Abstract: A series of results from the NeuroAI over the past fifteen years have raised core questions both about how to compare Deep Neural Network (DNN) models to the brain, and about how much convergent evolution to expect between artificial networks and real brain networks.
Related stories
arXiv AI
Jun 4
A Unified Geometric Space for Topological Alignment Between Transformer-Based Models and Human Brain Networks
arXiv:2510. 24342v2 Announce Type: replace Abstract: Prior brain-AI alignment studies are typically constrained by specific inputs and tasks, limiting their ability to capture organizational properties across models with different modalities.
arXiv Machine Learning
Jun 24
Similarity of Neural Network Representations in Superposition
arXiv:2604. 00208v2 Announce Type: replace Abstract: Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features.
arXiv Machine Learning
Jun 4
Beyond Structural Symmetries: Linear Mode Connectivity via Neuron Identifiability
arXiv:2606. 04754v1 Announce Type: new Abstract: Many striking phenomena in deep learning, such as linear mode connectivity and the structured behavior of training dynamics, are closely tied to parameter symmetries: transformations that leave the realized function unchanged.
arXiv Machine Learning
Jun 18
InTrain: Intrinsic Trainability for Zero-Cost Neural Architecture Search
arXiv:2606. 18676v1 Announce Type: new Abstract: Training-free neural architecture search promises efficient discovery of high-performance networks without costly training.
arXiv Machine Learning
Jul 30
Automorphism-Induced Non-Canonicity in Top-k Explanations of Graph Neural Networks
arXiv:2607. 26344v1 Announce Type: new Abstract: A gradient-based GNN explainer given a molecule with two chemically equivalent nitro groups assigns them attribution scores that are equal to the last bit.
arXiv AI
Jun 16
No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks
arXiv:2405. 02369v2 Announce Type: replace-cross Abstract: In the past decade, many successful networks are on novel architectures, which almost exclusively use the same type of neurons.
arXiv Machine Learning
Jul 3
From Approximation to Emergence: A Theory of Deep Learning
arXiv:2607. 01311v1 Announce Type: new Abstract: Deep learning has outgrown any single mathematical explanation.
arXiv Machine Learning
Jun 10
Overcoming Rank Collapse in Feedback Alignment
arXiv:2606. 11123v1 Announce Type: new Abstract: Backpropagation (BP) is widely viewed as biologically implausible, in part because it requires feedback weights to be the transpose of forward weights for error propagation.
arXiv Machine Learning
Jul 13
How are linear representations learned? Exact solutions to the dynamics of abstraction
arXiv:2607. 08843v1 Announce Type: new Abstract: In artificial and biological neural networks, concepts are often encoded as consistent linear directions in representation space.
arXiv Machine Learning
1d ago
On the Principles Behind Neural Network Optimizers
arXiv:2608. 16760v1 Announce Type: new Abstract: Reliable optimization is central to neural network (NN) training, yet Adam, the default optimizer for modern LLMs, rests on a fragile foundation.
arXiv Machine Learning
Aug 10
Mathematical Principles and Experimental Discoveries of the Emergence of Symbolic Patterns in Artificial Neural Networks
arXiv:2608. 06839v1 Announce Type: new Abstract: Artificial Neural networks (ANNs) are often treated as black-box models, making explainability a central challenge in deep learning.