arXiv:2608. 13335v1 Announce Type: new Abstract: Neural networks trained by gradient descent on a smooth cost function can nevertheless learn in steps: the cost holds on long plateaus and then drops abruptly.
arXiv:2605. 05113v2 Announce Type: replace Abstract: We study signal propagation in linear recurrent models at finite width.
arXiv:2610. 31663v1 Announce Type: cross Abstract: Constructing accurate and computationally tractable approximation spaces is a fundamental task in scientific computing.
arXiv:2608. 14638v1 Announce Type: new Abstract: In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points.
arXiv:2610. 22191v1 Announce Type: cross Abstract: In Hebbian learning from examples a Hopfield network of $N$ neurons is not supplied with the $K$ archetypes it has to retrieve but with $M$ blurred examples of each, of quality $r$, and whether it can infer them depends on the load $\alpha=K/N$ (interference among them) and on the dataset entropy $\rho=(1-r^2)/(Mr^2)$ (how little the examples say about them).
arXiv:2607. 03347v1 Announce Type: new Abstract: We consider the Multiscale Single-Index Model (MSIM), first introduced in \cite{oymak2021learning}, as a stylized model for hierarchical learning with \emph{scale separation}.
arXiv:2609.37745v1 Announce Type: new Abstract: Neural scaling, in which loss falls as a power law with training, is central to large language models, and one recent proposal is that a $1/3$ exponent...
arXiv:2609. 25710v1 Announce Type: cross Abstract: The statistical accuracy of neural networks depends on both their approximation power and the complexity of the class fitted from data.
The edge-of-chaos condition preserves first-order input perturbations in wide randomly initialized networks, but physics-informed losses, score matching and derivative regularization depend on higher...
arXiv:2605.12049v2 Announce Type: replace-cross Abstract: Cortical neurons are complex, multi-timescale processors wired into recurrent circuits, shaped by long evolutionary pressure under stringent...
arXiv:2609.09244v1 Announce Type: new Abstract: The edge-of-chaos condition preserves first-order input perturbations in wide randomly initialized networks, but physics-informed losses, score matchin...
The paper introduces Sliding Networks, a family of residual neural network structures that aim to preserve both feature learning and feature diversity while scaling width and depth. By maintaining a limiting ratio of residual width to the product of hidden width and depth, the authors derive a joint scaling recipe that converges to a stochastic differential equation (SDE) limit with locally maximal feature learning. Experiments on residual-MLP and language models demonstrate that this scaling approach allows redistribution of computational resources between width and depth, though performance varies when parameters are regrouped at a fixed count.