arXiv Machine Learning
Jul 8
Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity
arXiv:2607. 05546v1 Announce Type: cross Abstract: We develop a unified function space theory of deep fully connected neural networks.
Compositionality Emerges in a Narrow Depth-Connectivity Regime: Architecture Constraints and Solution Manifolds
arXiv:2606. 19941v1 Announce Type: new Abstract: Compositionality is believed to be the foundation for generalization, enabling models to reuse meaningful primitives in novel combinations.
Related stories
arXiv Machine Learning
Jun 18
InTrain: Intrinsic Trainability for Zero-Cost Neural Architecture Search
arXiv:2606. 18676v1 Announce Type: new Abstract: Training-free neural architecture search promises efficient discovery of high-performance networks without costly training.
arXiv Machine Learning
Jun 5
Gradient Descent with Large Step Size Restores Symmetry in Deep Linear Networks with Multi-Pathway
arXiv:2606. 05219v1 Announce Type: new Abstract: Recent analyses of multi-pathway Deep Linear Networks use Gradient Flow to predict a "winner-takes-all" specialization in which path symmetry breaks and each feature concentrates in a single pathway.
arXiv Machine Learning
Jul 7
Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models
arXiv:2602. 24264v2 Announce Type: replace-cross Abstract: Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems.
arXiv Machine Learning
Jul 10
Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks
arXiv:2607. 07884v1 Announce Type: new Abstract: In this short note we consider the gradient descent dynamics of deep scalar linear networks, $f(x) = \prod_{l=1}^L w_l x$, which enjoy exact time-course solutions for any integer depth.
arXiv AI
Jul 16
Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth
arXiv:2607. 14018v1 Announce Type: cross Abstract: We investigate how each component of the Transformer feedforward block architecture design determines how much rank survives across depth at initialization.
arXiv Machine Learning
Jun 24
Layer-wise Geometric Approximation Rates for Deep Networks
arXiv:2604. 20219v2 Announce Type: replace Abstract: Depth is widely viewed as a central contributor to the success of deep neural networks, whereas standard neural network approximation theory typically provides guarantees only for the final output and leaves the role of intermediate layers largely unclear.
Hugging Face Trending Papers
Jun 22
Sublinearly Structured Deep Neural Networks Achieve Feature Learning Consistency for Compositional Functions
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
arXiv Machine Learning
Aug 11
TEMPER: Tensorized Efficient Manifold-constrained Parameterization for Expressive Residual Routing
arXiv:2608. 07851v1 Announce Type: new Abstract: Residual connections rely on a static residual pathway, and are essential for training deep neural networks.
arXiv Machine Learning
Jul 1
Revisiting the Volume Hypothesis
arXiv:2606. 31282v1 Announce Type: new Abstract: Modern deep neural networks often contain far more parameters than needed to fit their training data, yet they achieve impressive generalization.
arXiv Machine Learning
Jun 5
On Universality of Deep Equivariant Networks
arXiv:2510. 15814v2 Announce Type: replace-cross Abstract: Universality results for equivariant neural networks remain rare.
arXiv Machine Learning
Jun 16
The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints
arXiv:2606. 16028v1 Announce Type: new Abstract: Modern deep learning architectures are increasingly multi-task and multi-modal, using a pretrained foundation model combined with task-specific, fine-tuned models.