arXiv AI

ELiSe: Efficient Learning of Sequences in Structured Recurrent Networks

The paper introduces ELiSe, a model that leverages cortical network scaffolds and dendritic compartments to learn complex non‑Markovian spatio‑temporal patterns using only local, always‑on, phase‑free synaptic plasticity. It demonstrates the model’s ability to acquire and replay intricate sequences, exemplified by a birdsong learning mock‑up, and shows robustness to external disturbances and flexibility in parameter settings.

arXiv Machine Learning
Oct 9

Learning infinite context windows in recurrent architectures via spatial neural computing

The paper proposes a second‑order recurrent neural network that replaces traditional neuron‑to‑neuron communication with a spatially evolving field governed by discretized partial differential equations. This design creates a structured infinite‑order RNN where each state depends on its entire history, providing an unbounded receptive field while keeping the number of parameters fixed. The authors derive conditions for marginal stability that prevent vanishing or exploding gradients, and demonstrate that the architecture outperforms other recurrent models on long‑horizon benchmarks with fewer parameters.

By Aleix Salvador-Pomarol, Arthur N. Montanari, Earl K. Miller, Adilson E. Motter, Jorge Cort\'es
arXiv Machine Learning
Jun 25

Frequency Domain Reservoir Computing

arXiv:2606. 24969v1 Announce Type: new Abstract: While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states.

By Klaus Schertler, Xiomara Runge, Andrea Ceni, David Kappel, Claudio Gallicchio
arXiv Machine Learning
Aug 6

Echo Flow Networks

arXiv:2509. 24122v3 Announce Type: replace Abstract: At the heart of time-series forecasting (TSF) lies a fundamental challenge: how can models efficiently and effectively capture long-range temporal dependencies across ever-growing sequences?

By Hongbo Liu, Jia Xu
arXiv Computer Vision
Oct 16

The Dormant Spiking Neuron: A State-Driven Mechanism for Efficient Spiking Neural Networks

The paper introduces the Dormant Spiking Neuron (DSN), a state-driven mechanism that allows neurons in spiking neural networks to enter a dormant state when their membrane potential falls below a negative threshold. This dynamic gating prunes redundant computations at the instance level and is trained end‑to‑end via backpropagation through time with surrogate gradients. Experiments on static and neuromorphic datasets, including ImageNet, show that DSN reduces computational energy by 40% to over 70% and improves baseline SNN accuracy by 1.34%, outperforming state‑of‑the‑art static pruning methods.

By Ruichen Ma, Yicai Chen, Pujun Zhou, Yue Zuo, Chongyang Li, Shuang Liu, Shaogang Hu, Guanchao Qiao
arXiv AI
1d ago

A Variational Latent Equilibrium for Learning in Neuronal Circuits

The paper proposes a biologically plausible framework to approximate backpropagation through time (BPTT) for learning complex temporal dependencies in neuronal circuits. It derives a time‑continuous equivalent of BPTT using a prospective energy function and introduces a learning rule for backward weights that ensures fully local space‑time dynamics, correcting gain errors inherent in previous approximations. This approach unifies and extends prior local, phase‑free credit assignment methods, offering a rigorous basis for spatiotemporal deep learning in the brain and guiding the design of physical circuits that implement these computations.

By Simon Brandt, Paul Haider, Walter Senn, Federico Benitez, Mihai A. Petrovici
arXiv AI
Oct 12

Does Learning Recurrent Networks Matter in Partially Observable RL?

The paper investigates whether a frozen recurrent network can provide sufficient memory for partially observable Markov decision processes (POMDPs). Using a fixed recurrent network combined with a trainable output module, the authors demonstrate that such networks can recall information over hundreds of steps and achieve competitive control performance across diverse POMDP tasks, while using fewer than half the trainable parameters of fully trainable recurrent networks. These findings suggest that learning the recurrent memory module is not always necessary, offering a simple, parameter‑efficient baseline for partially observable reinforcement learning.

By Zekang Wang, Zhe He, Gaojie Jin
arXiv Machine Learning
Jul 28

Frequency-Based Reservoir computing

arXiv:2607. 24420v1 Announce Type: cross Abstract: Reservoir computing has emerged as an efficient machine learning framework for predicting time series generated by dynamical systems.

By Arthur S Powanwe