The paper proposes a second‑order recurrent neural network that replaces traditional neuron‑to‑neuron communication with a spatially evolving field governed by discretized partial differential equations. This design creates a structured infinite‑order RNN where each state depends on its entire history, providing an unbounded receptive field while keeping the number of parameters fixed. The authors derive conditions for marginal stability that prevent vanishing or exploding gradients, and demonstrate that the architecture outperforms other recurrent models on long‑horizon benchmarks with fewer parameters.
By Aleix Salvador-Pomarol, Arthur N. Montanari, Earl K. Miller, Adilson E. Motter, Jorge Cort\'es
arXiv:2508. 21172v3 Announce Type: replace-cross Abstract: Echo State Networks (ESNs) are a particular type of untrained Recurrent Neural Networks (RNNs) within the Reservoir Computing (RC) framework, popular for their fast and efficient learning.
By Matteo Pinna, Andrea Ceni, Claudio Gallicchio
arXiv:2610.19997v1 Announce Type: new
Abstract: Efficient sequence modeling remains a fundamental challenge in machine learning. Despite the success of Transformers, recurrent neural networks (RNNs)...
By Mario Mallea, Angela Nebot, Franscisco Mugica
arXiv:2606. 24969v1 Announce Type: new Abstract: While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states.
By Klaus Schertler, Xiomara Runge, Andrea Ceni, David Kappel, Claudio Gallicchio
arXiv:2509. 24122v3 Announce Type: replace Abstract: At the heart of time-series forecasting (TSF) lies a fundamental challenge: how can models efficiently and effectively capture long-range temporal dependencies across ever-growing sequences?
By Hongbo Liu, Jia Xu
arXiv:2606. 28380v1 Announce Type: cross Abstract: The intricate structures of biological neural networks largely emerge during development, guided by a comparatively compressed blueprint encoded in the genome.
By Mani Hamidi, Sina Khajehabdollahi, Charley M. Wu, Emmanouil Giannakakis
The paper introduces the Dormant Spiking Neuron (DSN), a state-driven mechanism that allows neurons in spiking neural networks to enter a dormant state when their membrane potential falls below a negative threshold. This dynamic gating prunes redundant computations at the instance level and is trained end‑to‑end via backpropagation through time with surrogate gradients. Experiments on static and neuromorphic datasets, including ImageNet, show that DSN reduces computational energy by 40% to over 70% and improves baseline SNN accuracy by 1.34%, outperforming state‑of‑the‑art static pruning methods.
By Ruichen Ma, Yicai Chen, Pujun Zhou, Yue Zuo, Chongyang Li, Shuang Liu, Shaogang Hu, Guanchao Qiao
arXiv:2607. 02283v1 Announce Type: cross Abstract: In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures -- Transformers, Mamba, state-space models, and MLPs.
By Juwei Shen, Yujie Wu, Changwen Chen
The paper proposes a biologically plausible framework to approximate backpropagation through time (BPTT) for learning complex temporal dependencies in neuronal circuits. It derives a time‑continuous equivalent of BPTT using a prospective energy function and introduces a learning rule for backward weights that ensures fully local space‑time dynamics, correcting gain errors inherent in previous approximations. This approach unifies and extends prior local, phase‑free credit assignment methods, offering a rigorous basis for spatiotemporal deep learning in the brain and guiding the design of physical circuits that implement these computations.
By Simon Brandt, Paul Haider, Walter Senn, Federico Benitez, Mihai A. Petrovici
The paper investigates whether a frozen recurrent network can provide sufficient memory for partially observable Markov decision processes (POMDPs). Using a fixed recurrent network combined with a trainable output module, the authors demonstrate that such networks can recall information over hundreds of steps and achieve competitive control performance across diverse POMDP tasks, while using fewer than half the trainable parameters of fully trainable recurrent networks. These findings suggest that learning the recurrent memory module is not always necessary, offering a simple, parameter‑efficient baseline for partially observable reinforcement learning.
By Zekang Wang, Zhe He, Gaojie Jin
arXiv:2602. 01196v2 Announce Type: replace Abstract: Recurrent neural policies are widely used in partially observable control and meta-RL tasks.
By Jin Li, Yue Wu, Mengsha Huang, Yuhao Sun, Hao He, Xianyuan Zhan
arXiv:2607. 24420v1 Announce Type: cross Abstract: Reservoir computing has emerged as an efficient machine learning framework for predicting time series generated by dynamical systems.
By Arthur S Powanwe