arXiv Machine Learning October 28, 2026 By John J. Vastola, Kanaka Rajan

A theory of noisy linear recurrent neural network learning dynamics

Read the original on arXiv Machine Learning →

The paper presents a theoretical study of how intrinsic noise affects learning dynamics in noisy linear recurrent neural networks. It demonstrates that noise can break degeneracies in the solution space, influence the alignment of read-in, readout, and recurrent weights, and even induce a phase transition where a non-degenerate solution becomes suboptimal under high noise and parameter constraints. These findings suggest that noise is not merely a nuisance but can fundamentally shape representation learning in RNNs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 2

Paradoxical noise preference in RNNs

arXiv:2601. 04539v2 Announce Type: replace-cross Abstract: In recurrent neural networks (RNNs) used to model biological neural networks, noise is typically introduced during training to emulate biological variability and regularize learning.

By Noah Eckstein, Manoj Srinivasan

safety

arXiv AI

Oct 19

Learning reveals invisible structure in low-rank RNNs

arXiv:2605. 04115v2 Announce Type: replace-cross Abstract: Learning in neural systems arises from synaptic changes that reshape the representations underlying behavior.

By Yoav Ger, Omri Barak

arXiv Machine Learning

Jul 31

Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks

arXiv:2607. 28185v1 Announce Type: new Abstract: Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional subspace.

By Mostafa Haghir Chehreghani

Hugging Face Trending Papers

Jul 30

Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks

Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional subspace. This phenomenon limits the effective depth of message-passing architectures and motivates the search for mechanisms that preserve representation diversity.

arXiv Machine Learning

Jun 4

Drift-Diffusion Matching: Embedding dynamics in latent manifolds of asymmetric neural networks

arXiv:2602. 14885v2 Announce Type: replace-cross Abstract: Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that restricts network dynamics to gradient-like flows.

By Ram\'on Nartallo-Kaluarachchi, Renaud Lambiotte, Alain Goriely

rag agents diffusion

arXiv Machine Learning

Jun 30

Scalar Representations of Neural Network Training Dynamics

arXiv:2606. 30384v1 Announce Type: new Abstract: Training in artificial neural networks can be viewed as a trajectory evolving through a high-dimensional loss landscape.

By Pedro Jim\'enez-Gonz\'alez, Miguel C. Soriano, Lucas Lacasa

rag

One email a morning, machine-written

Related stories

Paradoxical noise preference in RNNs

Learning reveals invisible structure in low-rank RNNs

Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks

Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks

Drift-Diffusion Matching: Embedding dynamics in latent manifolds of asymmetric neural networks

Scalar Representations of Neural Network Training Dynamics