arXiv Machine Learning
Jun 5
Pretraining Recurrent Networks without Recurrence
arXiv:2606. 06479v1 Announce Type: new Abstract: Training recurrent neural networks (RNNs) requires assigning credit across long sequences of computations.
Learning Long-Range Dependencies with Temporal Predictive Coding
arXiv:2602. 18131v2 Announce Type: replace Abstract: Temporal Predictive Coding provides a layer-local, parallelisable mechanism for learning in recurrent systems, making it an attractive candidate for online local learning on neuromorphic and edge hardware.
Related stories
arXiv AI
Jun 2
SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition
arXiv:2606. 00732v1 Announce Type: new Abstract: Learning long-range non-stationary temporal patterns remains a core challenge for modern sequence models, particularly in strict streaming settings.
arXiv Machine Learning
Jun 5
MSTN: A Lightweight and Fast Model for General TimeSeries Analysis
arXiv:2511. 20577v5 Announce Type: replace Abstract: Real-world time series often exhibit strong non-stationarity, complex nonlinear dynamics, and behavior expressed across multiple temporal scales, from rapid local fluctuations to slow-evolving long-range trends.
arXiv Machine Learning
Aug 10
Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction
arXiv:2608. 07420v1 Announce Type: new Abstract: World models are expected to support imagination over extended temporal horizons, yet most are still trained through local few-step prediction objectives and deployed by recursively rolling out their own predictions.
arXiv Machine Learning
Jun 25
Frequency Domain Reservoir Computing
arXiv:2606. 24969v1 Announce Type: new Abstract: While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states.
arXiv AI
Jul 9
Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization
arXiv:2604. 01577v3 Announce Type: replace-cross Abstract: We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory.
arXiv AI
Jun 16
Learning in the Recurrent State: Gradient Descent with Linear Recurrent Networks
arXiv:2410. 11687v3 Announce Type: replace-cross Abstract: Linear recurrent networks (LRNNs) offer linear-time sequence modeling, but standard recurrent updates do not directly expose the supervised products needed for in-context gradient descent.
Hugging Face Trending Papers
Jul 2
DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learning
Dynamic expansion methods for class-incremental learning (CIL) protect task-specific knowledge by growing dedicated tokens or subnetworks, yet our analyses suggest that classification supervision alone does not sufficiently preserve task-agnostic shared backbone representations over long incremental sequences. We identify two intertwined challenges: cross-task confusion from sequential training on predominantly current-task data, which biases decision boundaries toward recent tasks; and under-optimized shared representations in the backbone that cap long-term discriminability as tasks accumulate.
arXiv Machine Learning
Aug 6
Echo Flow Networks
arXiv:2509. 24122v3 Announce Type: replace Abstract: At the heart of time-series forecasting (TSF) lies a fundamental challenge: how can models efficiently and effectively capture long-range temporal dependencies across ever-growing sequences?
arXiv Machine Learning
Jun 5
Incremental Transformer Neural Processes
arXiv:2602. 18955v2 Announce Type: replace Abstract: Neural Processes (NPs), and specifically Transformer Neural Processes (TNPs), have demonstrated remarkable performance across tasks ranging from spatiotemporal forecasting to tabular data modelling.
arXiv AI
Jun 4
MesaNet: Sequence Modeling by Locally Optimal Test-Time Training
arXiv:2506. 05233v2 Announce Type: replace-cross Abstract: Sequence modeling is currently dominated by causal transformer architectures that use softmax self-attention.
arXiv Machine Learning
5d ago
MARCH: Scaling Recurrent Memory with Content-Routed State Anchors
arXiv:2608. 12435v1 Announce Type: new Abstract: Transformers owe much of their strong long-context retrieval capability to a token-level memory that grows with context length.