arXiv Machine Learning
Do Neural Models Prefer Their Submodels?
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 22
Functional Equivalence and Geometric Diversity in Neural Network Approximations: An Empirical Characterization
arXiv:2607. 18930v1 Announce Type: cross Abstract: The Universal Approximation Theorem states that a neural network with a single hidden layer is sufficient to approximate any continuous univariate function on a compact domain to arbitrary error.
Hugging Face Trending Papers
Jul 21
Functional Equivalence and Geometric Diversity in Neural Network Approximations: An Empirical Characterization
The Universal Approximation Theorem states that a neural network with a single hidden layer is sufficient to approximate any continuous univariate function on a compact domain to arbitrary error. However, the uniqueness of such neural network representations is not guaranteed, raising questions about practical identifiability.
arXiv AI
Jun 4
A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks
arXiv:2606. 04327v1 Announce Type: cross Abstract: We investigate the geometric structure of stationary plateaus that arise in the loss landscape of two-layer neural networks with smooth activation functions.
arXiv Machine Learning
Oct 23
Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions
arXiv:2610. 31264v1 Announce Type: new Abstract: We construct a continuous function on $[-1,1]^2$ for which the infimum of the $L^2$-approximation error over the class of ReLU networks with two hidden layers and architecture $(2,2,1,1)$ is not attained.
arXiv Machine Learning
Sep 11
Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs
The paper investigates how multilayer perceptrons (MLPs) learn features in regression tasks with clustered data. It finds that instead of forming a single global low‑dimensional representation, MLPs develop monosemantic specialized neurons—each neuron aligns strongly with a specific predictive feature relevant to a particular region of the input space. This specialization results in a collection of local low‑dimensional representations, giving MLPs a provable data‑efficiency advantage over methods that rely on a global representation.
arXiv Statistics ML
Oct 19
A Class of R\'enyi-based Concentration Metrics for Kolmogorov-Arnold Geometry in Multilayer Perceptrons
The paper introduces a new family of concentration metrics, called α-RC, based on Rényi divergence to assess how the magnitudes of Jacobian minors in multilayer perceptrons are distributed. Unlike the existing Participation Ratio, these metrics are bounded between 0 and 1 regardless of network size, with 0 meaning uniform spread and 1 indicating dominance by a single minor. For α=2, the α-RC metric reduces to a transformed version of the Participation Ratio, and experiments on single-hidden-layer MLPs demonstrate its practical advantages.