LAIR-Net is a Leaky Alignment-Impulse Residual Network designed for tabular regression that incorporates a shallow learned anchor into each hidden state via a leaky residual transition. The authors provide a depth‑uniform bound on input‑perturbation sensitivity and show through controlled simulations that performance gains over randomized baselines stem from the anchor rather than recursion or added capacity. On 23 benchmark datasets, LAIR‑Net achieves the best average rank among eight randomized networks and twelve conventional models, with its success linked to the presence of nonlinear target structure and manageable noise levels.
Backpropagation (BP) remains the predominant algorithm for training deep neural networks, but it is biologically implausible and suffers from the backward locking problem, constraining parallelization...
arXiv:2606. 11123v1 Announce Type: new Abstract: Backpropagation (BP) is widely viewed as biologically implausible, in part because it requires feedback weights to be the transpose of forward weights for error propagation.
arXiv:2606. 04476v1 Announce Type: new Abstract: In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.
arXiv:2602. 07494v2 Announce Type: replace Abstract: Deeper modern architectures are costly to train, making hyperparameter transfer preferable to expensive repeated tuning.
arXiv:2605. 11911v2 Announce Type: replace Abstract: Predictive Coding (PC) is an influential account of cortical learning.
arXiv:2608. 04442v1 Announce Type: new Abstract: Robustness to natural corruptions remains a fundamental challenge for deep neural networks.
arXiv:2607. 24887v1 Announce Type: cross Abstract: Existing theories of neural-network width characterize asymptotic limits, but provide limited guidance on whether an expansion direction identified from finite training data remains beneficial on unseen data.
The paper introduces MATCHA, a method that learns explicit many-to-many layer maps between independently trained language models. MATCHA decomposes the alignment into a layer map—an explicit target‑by‑source matrix—and a shared feature map, allowing joint learning of layer correspondence and feature transformation from prompts. Experiments on 42 model pairs across three families show that MATCHA reconstructs target activations more faithfully and improves retrieval metrics, while enabling transfer of activation‑space interventions between models.
arXiv:2602.09009v2 Announce Type: replace-cross Abstract: Scaling network depth has been a central driver behind the success of modern foundation models, yet recent investigations suggest that deep l...
arXiv:2608. 14664v1 Announce Type: new Abstract: How can we determine whether a trained neural network is already deep enough?
arXiv:2607. 14018v1 Announce Type: cross Abstract: We investigate how each component of the Transformer feedforward block architecture design determines how much rank survives across depth at initialization.
arXiv:2606. 16112v1 Announce Type: cross Abstract: Residual architectures are ubiquitous in deep learning, but they suffer from a subtle structural limitation: the norm of the residual stream can grow rapidly with depth.