The paper introduces a new family of concentration metrics, called α-RC, based on Rényi divergence to assess how the magnitudes of Jacobian minors in multilayer perceptrons are distributed. Unlike the existing Participation Ratio, these metrics are bounded between 0 and 1 regardless of network size, with 0 meaning uniform spread and 1 indicating dominance by a single minor. For α=2, the α-RC metric reduces to a transformed version of the Participation Ratio, and experiments on single-hidden-layer MLPs demonstrate its practical advantages.
arXiv:2609.32503v2 Announce Type: replace Abstract: Kolmogorov-Arnold Networks (KANs) are motivated in part by interpretability: their learned edge functions can be inspected, pruned, and reduced to...
arXiv:2606. 17886v1 Announce Type: new Abstract: Monotonicity has been a long-running architectural inductive bias for neural networks, motivated by tabular, scientific, and economic settings where outputs are known to respond monotonically to certain inputs.
arXiv:2607. 18930v1 Announce Type: cross Abstract: The Universal Approximation Theorem states that a neural network with a single hidden layer is sufficient to approximate any continuous univariate function on a compact domain to arbitrary error.
Monotonicity has been a long-running architectural inductive bias for neural networks, motivated by tabular, scientific, and economic settings where outputs are known to respond monotonically to certain inputs. Existing approaches are MLP- or flow-based and lack per-edge functional transparency; the only Kolmogorov--Arnold Network (KAN) variant with monotonicity, MonoKAN, enforces the constraint only on a restricted parameter subset and requires a projection-style training procedure.
arXiv:2605.12049v2 Announce Type: replace-cross Abstract: Cortical neurons are complex, multi-timescale processors wired into recurrent circuits, shaped by long evolutionary pressure under stringent...
The Universal Approximation Theorem states that a neural network with a single hidden layer is sufficient to approximate any continuous univariate function on a compact domain to arbitrary error. However, the uniqueness of such neural network representations is not guaranteed, raising questions about practical identifiability.
The paper studies Kolmogorov‑Arnold Networks (KANs), a neural architecture that treats activation functions as learnable components, offering improved interpretability for scientific applications. It investigates how KANs scale with dataset size on image classification tasks (MNIST, Fashion‑MNIST) and a magnetic‑parameter regression task, revealing a broken neural scaling law that transitions from a faster to a slower decay of test loss as data grows. The authors also analyze how the learned activation functions evolve from simple linear approximations to more complex, interpretable symbolic forms as more data is provided.
arXiv:2605.10795v2 Announce Type: replace-cross Abstract: Large language models excel at factual recall, yet the fundamental limits of storing input--output associations with neural networks remain u...
arXiv:2605.15688v2 Announce Type: replace-cross Abstract: Concept Activation Vectors (CAV) and the Testing with CAVs (TCAV) method are fundamental tools for concept-based explainability in deep learn...
The paper investigates how the geometry of teacher neural networks affects the learnability of student networks in teacher‑student setups. By formalizing learnability as the success rate of reaching the global minimum, the authors identify two teacher distributions—one maximizing node dissimilarity (easy) and one minimizing it (hard)—that lead to markedly different success rates across various settings and activation functions. They analyze the loss landscape of small networks, revealing two types of suboptimal local minima (out‑of‑bounds and interior) whose attraction regions depend on teacher structure, and demonstrate that adjusting learning rates for the readout layer and inner biases can improve success rates. whyItMatters:"The study highlights that teacher geometry, often overlooked, plays a crucial role in determining how effectively a student network can learn, offering guidance for designing more realistic teacher‑student experiments."
The paper presents a near-complete, nonasymptotic generalization theory for multilayer neural networks using path regularization, applicable to broad Lipschitz loss functions without requiring bounded loss or extreme network hyperparameters. It provides an explicit upper bound that addresses approximation rates in generalized Barron spaces and demonstrates the double descent phenomenon for ReLU networks. The authors claim near-minimax optimality for regression problems and plan to establish matching lower bounds in future work.
The paper introduces a dimension‑free PAC‑Bayes generalization bound for deep neural networks by placing the prior over the non‑parametric space of data distributions using a Dirichlet Process. This shift eliminates the dependence on the number of network parameters, yielding a bound that remains non‑vacuous even for over‑parameterized models. Experiments on MNIST with up to 4.2 million parameters show the new bound staying stable at 8.8 %–9.4 %, whereas classical weight‑space bounds become vacuous.