← Back to all news
arXiv Machine Learning By Shivang Rawat, Mirko Morello, Flaviano Morone, David J. Heeger

Prospective Coding Improves Learning in Deep Continuous-Time Recurrent Networks

Read the original on arXiv Machine Learning →

The paper introduces Recursive Quadrature Filters (RQFs), complex‑valued temporal filters that act as band‑pass filters within diagonal state‑space models. By making each layer’s bottom‑up input prospective through a parameter‑free two‑tap update, the authors mitigate depth‑dependent gradient attenuation in deep continuous‑time recurrent networks. Experiments on RQFs, S5, and ORGaNICs show that prospective variants match or surpass non‑prospective controls, achieving high accuracy on raw‑audio Speech Commands and the Path‑X task with few parameters.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jun 25

Frequency Domain Reservoir Computing

arXiv:2606. 24969v1 Announce Type: new Abstract: While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states.

By Klaus Schertler, Xiomara Runge, Andrea Ceni, David Kappel, Claudio Gallicchio
llmsragbenchmarks
More like this →
arXiv Machine Learning
Aug 27

Gated Recurrent Transformers: Expressive Depth through Recurrent Modulation

The paper introduces Gated Recurrent Transformers, a depth‑sharing architecture that brackets a single shared core with fixed prelude and coda blocks and uses a lightweight projection and element‑wise update gate to modulate recurrent updates. This design allows functional specialization across recurrences while reducing memory footprint. Experiments show that, under equal FLOPs or parameter budgets, the recurrent model matches or surpasses deeper GPT‑2 Small baselines, achieving similar or better accuracy with fewer parameters and lower peak decoding memory.

By Amr Hegazy, Amr Alanwar, Mostafa Elhoushi
llmsefficiency
More like this →
arXiv Machine Learning
Aug 6

Echo Flow Networks

arXiv:2509. 24122v3 Announce Type: replace Abstract: At the heart of time-series forecasting (TSF) lies a fundamental challenge: how can models efficiently and effectively capture long-range temporal dependencies across ever-growing sequences?

By Hongbo Liu, Jia Xu
benchmarks
More like this →