The paper presents a theoretical study of how intrinsic noise affects learning dynamics in noisy linear recurrent neural networks. It demonstrates that noise can break degeneracies in the solution space, influence the alignment of read-in, readout, and recurrent weights, and even induce a phase transition where a non-degenerate solution becomes suboptimal under high noise and parameter constraints. These findings suggest that noise is not merely a nuisance but can fundamentally shape representation learning in RNNs.
arXiv:2601. 04539v2 Announce Type: replace-cross Abstract: In recurrent neural networks (RNNs) used to model biological neural networks, noise is typically introduced during training to emulate biological variability and regularize learning.
arXiv:2605. 04115v2 Announce Type: replace-cross Abstract: Learning in neural systems arises from synaptic changes that reshape the representations underlying behavior.
arXiv:2607. 28185v1 Announce Type: new Abstract: Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional subspace.
Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional subspace. This phenomenon limits the effective depth of message-passing architectures and motivates the search for mechanisms that preserve representation diversity.
arXiv:2602. 14885v2 Announce Type: replace-cross Abstract: Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that restricts network dynamics to gradient-like flows.
arXiv:2606. 30384v1 Announce Type: new Abstract: Training in artificial neural networks can be viewed as a trajectory evolving through a high-dimensional loss landscape.
Training in artificial neural networks can be viewed as a trajectory evolving through a high-dimensional loss landscape. However, the large number of trainable parameters makes the direct analysis of these dynamics challenging.
The paper proposes a second‑order recurrent neural network that replaces traditional neuron‑to‑neuron communication with a spatially evolving field governed by discretized partial differential equations. This design creates a structured infinite‑order RNN where each state depends on its entire history, providing an unbounded receptive field while keeping the number of parameters fixed. The authors derive conditions for marginal stability that prevent vanishing or exploding gradients, and demonstrate that the architecture outperforms other recurrent models on long‑horizon benchmarks with fewer parameters.
arXiv:2606. 10530v1 Announce Type: cross Abstract: Recent developments in brain recording are driving a demand for machine learning tools capable of decoding the latent structure of large populations of neurons.
arXiv:2511. 06609v4 Announce Type: replace Abstract: The accurate forecasting of complex, high-dimensional dynamical systems from observational data is a fundamental task across numerous scientific and engineering disciplines.
arXiv:2609.38356v1 Announce Type: new Abstract: Dynamical Systems Reconstruction (DSR) aims to infer models from observed time series that reproduce a system's qualitative long-term behavior. Continu...
arXiv:2507.10383v5 Announce Type: replace-cross Abstract: Recurrent neural networks are canonical models of biological memory. In these models, memories are represented by distributed patterns of neu...