Parallel Layer Normalization for Universal Approximation

The paper studies how grouping Layer Normalization (LN) units influences the approximation capacity of neural networks as their width increases. It shows that a shallow LN-Net with a single LN unit cannot approximate arbitrary functions regardless of width, but Parallel Layer Normalization networks (PLN-Nets) become universal as the number of parallel groups increases. The authors extend this result to deep networks, RMSNorm variants, feed‑forward and convolutional architectures, and validate the practical benefits of PLN through controlled MLP and physics‑informed neural network experiments, while also demonstrating that group count, grouped axes, replacement site, and label structure jointly affect fitting and generalization in CNN and ViT settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.