The paper studies how grouping Layer Normalization (LN) units influences the approximation capacity of neural networks as their width increases. It shows that a shallow LN-Net with a single LN unit cannot approximate arbitrary functions regardless of width, but Parallel Layer Normalization networks (PLN-Nets) become universal as the number of parallel groups increases. The authors extend this result to deep networks, RMSNorm variants, feed‑forward and convolutional architectures, and validate the practical benefits of PLN through controlled MLP and physics‑informed neural network experiments, while also demonstrating that group count, grouped axes, replacement site, and label structure jointly affect fitting and generalization in CNN and ViT settings.
arXiv:2610. 21527v1 Announce Type: new Abstract: Layer normalization (LN) and root mean square normalization (RMSNorm) depend on their inputs and remain nonlinear at inference.
arXiv:2403.04545v4 Announce Type: replace Abstract: Scaling factors in residual branches have emerged as a prevalent method for boosting neural network performance, especially in normalization-free a...
arXiv:2402. 00152v5 Announce Type: replace Abstract: Constructing the architecture of a neural network is a challenging pursuit for the machine learning community, and the dilemma of whether to go deeper or wider remains a persistent question.
arXiv:2502. 07209v4 Announce Type: replace Abstract: Physics-Informed Neural Networks (PINNs) seek to solve partial differential equations (PDEs) with deep learning.
arXiv:2607. 09967v1 Announce Type: cross Abstract: Many neural networks operations have a multiplicative nature rather than additive: halving or doubling a norm are analogous relatively but require unequal optimization distances when taking linear steps.
arXiv:2606. 00130v2 Announce Type: replace-cross Abstract: Large deep neural networks are costly to store and deploy because inference must move and evaluate many parameters.
Backpropagation (BP) remains the predominant algorithm for training deep neural networks, but it is biologically implausible and suffers from the backward locking problem, constraining parallelization...
arXiv:2607. 23397v1 Announce Type: new Abstract: Hierarchical neural networks are widely used in artificial intelligence, yet their mathematical properties remain incompletely understood.
arXiv:2602. 07697v3 Announce Type: replace-cross Abstract: Predictive coding (PC) is a biologically plausible alternative to standard backpropagation (BP) that minimises an energy function with respect to network activities before updating weights.
arXiv:2511. 00044v3 Announce Type: replace Abstract: Physical neural networks (PNNs) are promising candidates for next-generation computing, but existing demonstrations remain several orders of magnitude smaller than modern digital neural networks, whose recent advances have been driven by rapid growth in trainable parameters.
arXiv:2505. 09710v4 Announce Type: replace Abstract: We investigate deep morphological neural networks (DMNNs), studying how changes in algebraic structure affect the expressivity and trainability of deep architectures.