arXiv:2608. 06766v1 Announce Type: cross Abstract: Training changes a network's predictions while allocating task-relevant structure across its internal units.
arXiv:2606. 04476v1 Announce Type: new Abstract: In this paper, we study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.
arXiv:2606. 05863v1 Announce Type: new Abstract: Grokking suggests that fitting the training data and learning a simple underlying rule may occur on different time scales.
arXiv:2607. 03613v1 Announce Type: new Abstract: We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression.
arXiv:2607. 10869v1 Announce Type: new Abstract: We study the population gradient flow of an infinitely wide two-layer neural network learning a misspecified single-index model in high dimension.
arXiv:2605. 15435v2 Announce Type: replace Abstract: Standard deep-learning pipelines usually choose the network architecture before training and keep it fixed throughout optimization.
arXiv:2607. 03671v1 Announce Type: cross Abstract: Models of complex systems often have many parameters, yet are constrained by far fewer experimentally accessible observables: similar activity can emerge from coordinated parameter changes.
arXiv:2602. 19799v2 Announce Type: replace-cross Abstract: Despite recent algorithmic advances, we still lack principled ways to leverage the well-documented rescaling symmetries in ReLU neural network parameters.
arXiv:2602. 20062v2 Announce Type: replace Abstract: Pretraining and fine-tuning are central stages in modern machine learning systems.
arXiv:2605. 12765v3 Announce Type: replace Abstract: Large Language Models memorize vast amounts of training data, raising concerns regarding privacy, copyright infringement, and safety.
arXiv:2602. 03846v2 Announce Type: replace-cross Abstract: We develop a continual learning method for pretrained models that \emph{requires no access to old-task data}, addressing a practical barrier in foundation model adaptation where pretraining distributions are often unavailable.
arXiv:2607. 13432v1 Announce Type: new Abstract: Plasticity -- a neural network's ability to adapt to new tasks -- is critical for continual and transfer learning.
arXiv:2606. 30444v1 Announce Type: cross Abstract: Neural networks are known to be susceptible to over-reliance on spurious correlations.