The paper introduces the Recurrent Divisive Normalization Network (RDNN), a minimal model that incorporates divisive normalization—a common neural computation—to stabilize continuous working memory representations. Dynamical systems analysis shows that this biophysical constraint enables the network to converge to robust, high‑fidelity slow manifolds, while gradient dynamics during Backpropagation Through Time reveal an activity‑dependent local scaling that compresses the network’s effective rank into a low‑dimensional subspace. Ablation studies confirm that divisive normalization, rather than subtractive inhibition, is essential for preventing manifold shattering under time‑varying inputs.
By Zhaotian Gu, Jie Su, Weiwei Wang, Chang Liu, Tianyi Qian, Dahui Wang
The paper investigates how gradient descent behaves near codimension‑one bifurcations in recurrent neural networks by analyzing the global empirical Neural Tangent Kernel (GeNTK). Under local center‑manifold conditions, the parameter‑to‑state Jacobian is approximated by a low‑rank normal‑form operator, causing the GeNTK and Fisher information matrix to become strongly amplified and anisotropic, concentrating on a rank‑one or rank‑two channel depending on the bifurcation type. Experiments on high‑dimensional RNNs confirm that this low‑rank concentration coincides with abrupt loss changes, subtask interference, and aligns with changes in memory dynamics in a 15‑task LeakyRNN.
By James Hazelden, Eric Shea-Brown
arXiv:2606. 14975v1 Announce Type: cross Abstract: How the wiring and functional organization of cortex shape recurrent computation remains a central question in both neuroscience and machine learning.
By Mo Shakiba, Rana Rokni, Mohammad Mohammadi, Nima Dehghani
Training in artificial neural networks can be viewed as a trajectory evolving through a high-dimensional loss landscape. However, the large number of trainable parameters makes the direct analysis of these dynamics challenging.
arXiv:2606. 30384v1 Announce Type: new Abstract: Training in artificial neural networks can be viewed as a trajectory evolving through a high-dimensional loss landscape.
By Pedro Jim\'enez-Gonz\'alez, Miguel C. Soriano, Lucas Lacasa
arXiv:2511. 13899v2 Announce Type: replace-cross Abstract: Low-rank recurrent neural networks (lrRNNs) are a class of models that uncover low-dimensional latent dynamics underlying neural population activity.
By Chengrui Li, Yunmiao Wang, Yule Wang, Weihan Li, Dieter Jaeger, Anqi Wu
arXiv:2607. 10430v1 Announce Type: cross Abstract: Dimensionality reduction has proven powerful for identifying neural manifolds, which are low-dimensional structures underlying high-dimensional neural activity.
By Hardik Rajpal, Dan Goodman
arXiv:2512. 12225v3 Announce Type: replace Abstract: Developing artificial agents that unify representation, memory, adaptation, and prediction remains a fundamental challenge in artificial intelligence.
By Laha Ale
The paper introduces ELiSe, a model that leverages cortical network scaffolds and dendritic compartments to learn complex non‑Markovian spatio‑temporal patterns using only local, always‑on, phase‑free synaptic plasticity. It demonstrates the model’s ability to acquire and replay intricate sequences, exemplified by a birdsong learning mock‑up, and shows robustness to external disturbances and flexibility in parameter settings.
By Laura Kriener, Kristin V\"olk, Ben von H\"unerbein, Federico Benitez, Walter Senn, Mihai A. Petrovici
arXiv:2608.20682v1 Announce Type: new
Abstract: This paper formalizes and systematically characterizes Aristotelian Manifolds, a generalized structural framework built upon the Platonic Representatio...
By Michael Karnes, Alper Yilmaz
arXiv:2606. 02993v1 Announce Type: new Abstract: Understanding how structured internal structure emerges during neural network training is central to the study of deep learning.
By Jianliang He, Leda Wang, Fengzhuo Zhang, Siyu Chen, Zhuoran Yang
arXiv:2609.40292v1 Announce Type: new
Abstract: How is computation organized and reused across tasks and time in a trained recurrent network? Most analyses emphasize the geometry of neural activity,...
By James Hazelden