The paper investigates whether a frozen recurrent network can provide sufficient memory for partially observable Markov decision processes (POMDPs). Using a fixed recurrent network combined with a trainable output module, the authors demonstrate that such networks can recall information over hundreds of steps and achieve competitive control performance across diverse POMDP tasks, while using fewer than half the trainable parameters of fully trainable recurrent networks. These findings suggest that learning the recurrent memory module is not always necessary, offering a simple, parameter‑efficient baseline for partially observable reinforcement learning.
By Zekang Wang, Zhe He, Gaojie Jin
The paper examines whether a frozen recurrent network can supply sufficient memory for partially observable reinforcement learning tasks, instead of training the recurrent module jointly with the policy. Using a fixed recurrent network paired with a trainable output layer, the authors test this setup on various POMDPs with different memory demands, observation types, and action spaces. Results show that frozen networks can recall information over hundreds of steps and achieve competitive performance while using fewer than half the trainable parameters of fully trainable recurrent models.
arXiv:2609.38356v1 Announce Type: new
Abstract: Dynamical Systems Reconstruction (DSR) aims to infer models from observed time series that reproduce a system's qualitative long-term behavior. Continu...
By Sima Hashemi, Daniel Durstewitz, Georgia Koppe
arXiv:2410.14606v3 Announce Type: replace
Abstract: Learning from a stream of experience as it arrives, also known as streaming learning, is a core part of natural learning. However, reliable streami...
By Mohamed Elsayed, Elena Sorina Lupu, Gautham Vasan, A. Rupam Mahmood
arXiv:2609.38598v1 Announce Type: new
Abstract: Partially observable environments pose a fundamental challenge in deep reinforcement learning, requiring agents to compress temporal information from o...
By Sathya Kamesh Bhethanabhotla, Efstratios Gavves, Andr\'e Biedenkapp
arXiv:2604. 01577v3 Announce Type: replace-cross Abstract: We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory.
By Shota Takashiro, Masanori Koyama, Takeru Miyato, Yusuke Iwasawa, Yutaka Matsuo, Kohei Hayashi
arXiv:2609.33848v2 Announce Type: replace
Abstract: Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of m...
By Weiqi Wang, Yuxin Zhou, Mouxiang Chen, Siyuan Zhang, Yi Zhang, Yuyan Luo, Zhiyu Yin, Chencan Wu, Jiemin Jiang, Wentao Yao, Chujie Zheng, JianWei Zhang
ReHarness is a recurrent harness framework designed for agentic continual learning, enabling language model agents to adapt online to distribution shifts. It augments forward read/write operations with backward revision, using feedback from each instance’s trajectory to refine the harness state. Experiments on three continual learning benchmarks and multiple language model backbones show that ReHarness outperforms baseline methods, producing source‑grounded revisions that improve stability and plasticity.
By Yuxin Xiao, Zhiyun Lu, Jianyu Wang, Yangtian Zhang, Yang Song, Qifan Wang, Bailin Wang
arXiv:2610.00592v1 Announce Type: cross
Abstract: In partially observable reinforcement learning (RL), a later observation can make stored information obsolete or change what it implies for the next...
By Oleg Shchendrigin, Egor Cherepanov, Aleksandr I. Panov, Alexey K. Kovalev
arXiv:2605. 06384v3 Announce Type: replace-cross Abstract: We introduce MinMax Recurrent Neural Cascades (MinMax RNCs), a class of recurrent neural networks built from a novel form of recurrence over the MinMax algebra.
By Alessandro Ronca
arXiv:2602. 05999v3 Announce Type: replace Abstract: How does the amount of compute available to a reinforcement learning (RL) policy affect its learning?
By Raj Ghugare, Micha{\l} Bortkiewicz, Alicja Ziarko, Benjamin Eysenbach
arXiv:2609.35750v2 Announce Type: replace-cross
Abstract: Scaling the horizon of agentic LLMs is bottlenecked by the need to fit ever longer context traces in GPU memory. Context compaction has been...
By Emiliano Penaloza, Dane Malenfant, Dheeraj Vattikonda, Roger Creus Castanyer, Siddarth Venkatraman, Abhay Puri, Jonathan Light, Matthew James Sargent, Augustine N. Mavor-Parker, Massimo Caccia, Lucas Caccia, Glen Berseth, Esmeralda S. Whitammer, Alessandro Sordoni, Minseon Kim, Marc-Alexandre C\^ot\'e, Laurent Charlin, Guillaume Lajoie