arXiv Machine Learning

State Space Models Naturally Produce Time Cell and Oscillatory Behaviors and Scale to Abstract Cognitive Functions

arXiv Machine Learning
Oct 9

Learning infinite context windows in recurrent architectures via spatial neural computing

The paper proposes a second‑order recurrent neural network that replaces traditional neuron‑to‑neuron communication with a spatially evolving field governed by discretized partial differential equations. This design creates a structured infinite‑order RNN where each state depends on its entire history, providing an unbounded receptive field while keeping the number of parameters fixed. The authors derive conditions for marginal stability that prevent vanishing or exploding gradients, and demonstrate that the architecture outperforms other recurrent models on long‑horizon benchmarks with fewer parameters.

By Aleix Salvador-Pomarol, Arthur N. Montanari, Earl K. Miller, Adilson E. Motter, Jorge Cort\'es
arXiv AI
Jun 24

Topological Neural Dynamics: A Neuron-wise Framework for Sequence Modeling

arXiv:2606. 21295v2 Announce Type: replace-cross Abstract: Existing sequence models, including RNNs, LSTMs, continuous-time networks, and Transformers, share a common structural principle: layer-wise dynamics, where all neurons in the same layer co-evolve through a shared parameterized operator, leaving individual neurons no freedom to evolve independently.

By Borui Cai, Yao Zhao
arXiv AI
Sep 25

ELiSe: Efficient Learning of Sequences in Structured Recurrent Networks

The paper introduces ELiSe, a model that leverages cortical network scaffolds and dendritic compartments to learn complex non‑Markovian spatio‑temporal patterns using only local, always‑on, phase‑free synaptic plasticity. It demonstrates the model’s ability to acquire and replay intricate sequences, exemplified by a birdsong learning mock‑up, and shows robustness to external disturbances and flexibility in parameter settings.

By Laura Kriener, Kristin V\"olk, Ben von H\"unerbein, Federico Benitez, Walter Senn, Mihai A. Petrovici
arXiv AI
Jun 18

Structured Representation Learning with Locally Linear Embeddings and Adaptive Feature Fusion

arXiv:2606. 18469v1 Announce Type: cross Abstract: Neuroscientific research has revealed that the brain encodes complex behaviors by leveraging structured, low-dimensional manifolds and dynamically fusing multiple sources of information through adaptive gating mechanisms.

By Somjit Nath, Jackson J Cone, Derek Nowrouzezahrai, Samira Ebrahimi Kahou
arXiv Machine Learning
Oct 5

Neural Data Needs Semantic Tokenization: Behavioral Events as Boundaries of Session-Transferable Tokens

The paper introduces Tokenization with States (TWS), a method that segments neural recordings into behaviorally meaningful states defined by task events and represents each state as a token based on population geometry, bypassing neuron- or session-specific embeddings. TWS successfully decodes movement from regime boundaries that carry no target information and transfers across species and recording modalities with only a linear probe. The approach demonstrates that tokens derived from shared population dynamics can generalize across sessions, outperforming per-neuron foundation models that fail to generalize.

By Sangyoon Bae, Jiook Cha
arXiv Machine Learning
Oct 9

Local Timescale Gates for Timescale-Robust Continual Spiking Neural Networks

Local Timescale Gating (LT‑Gate) is a neuron model that tracks fast and slow information in parallel and uses a learned gate to adjust their influence, enabling spiking neurons to preserve long‑term context while responding quickly. The approach includes a variance‑tracking regularization inspired by biological homeostasis and operates with local updates, making it compatible with neuromorphic hardware such as Intel’s Loihi chip. Empirical results show LT‑Gate improves accuracy and retention on sequential learning tasks, achieving about 51 % final accuracy versus 46 % for a recent Hebbian baseline and outperforming prior SNN methods.

By Devansh Deep, Ansh Tiwari, Ayush Chauhan