The paper examines whether a frozen recurrent network can supply sufficient memory for partially observable reinforcement learning tasks, instead of training the recurrent module jointly with the policy. Using a fixed recurrent network paired with a trainable output layer, the authors test this setup on various POMDPs with different memory demands, observation types, and action spaces. Results show that frozen networks can recall information over hundreds of steps and achieve competitive performance while using fewer than half the trainable parameters of fully trainable recurrent models.
arXiv:2609.38598v1 Announce Type: new
Abstract: Partially observable environments pose a fundamental challenge in deep reinforcement learning, requiring agents to compress temporal information from o...
By Sathya Kamesh Bhethanabhotla, Efstratios Gavves, Andr\'e Biedenkapp
arXiv:2605. 24709v2 Announce Type: replace Abstract: Streaming reinforcement learning has emerged as an online learning paradigm that conforms to the restrictions of natural learning agents that process data incrementally, i.
By Noah Farr, Aryaman Reddi, Carlo D'Eramo, Jan Peters
The paper introduces ELiSe, a model that leverages cortical network scaffolds and dendritic compartments to learn complex non‑Markovian spatio‑temporal patterns using only local, always‑on, phase‑free synaptic plasticity. It demonstrates the model’s ability to acquire and replay intricate sequences, exemplified by a birdsong learning mock‑up, and shows robustness to external disturbances and flexibility in parameter settings.
By Laura Kriener, Kristin V\"olk, Ben von H\"unerbein, Federico Benitez, Walter Senn, Mihai A. Petrovici
arXiv:2603. 23461v2 Announce Type: replace Abstract: We study reinforcement learning (RL) with linear function approximation in Markov Decision Processes (MDPs) satisfying \emph{linear Bellman completeness} -- a fundamental setting where the Bellman backup of any linear value function remains linear.
By Zakaria Mhammedi, Alexander Rakhlin, Nneka Okolo
arXiv:2605.07057v2 Announce Type: replace
Abstract: Online reinforcement learning (RL) relies on the Markov property for guaranteed performance, but real-world applications often lack well-defined st...
By Jiamin Xu, Jacqueline Maasch, Kyra Gan
The paper introduces memoroids, a monoid‑based framework that reformulates memory models like RNNs and Transformers for better handling of long sequences in reinforcement learning. It critiques traditional batching approaches in recurrent RL, identifies their shortcomings, and proposes a new batching method using memoroids that enhances sample efficiency, boosts return, and simplifies recurrent loss implementation.
By Steven Morad, Chris Lu, Ryan Kortvelesy, Stephan Liwicki, Jakob Foerster, Amanda Prorok
arXiv:2602. 01196v2 Announce Type: replace Abstract: Recurrent neural policies are widely used in partially observable control and meta-RL tasks.
By Jin Li, Yue Wu, Mengsha Huang, Yuhao Sun, Hao He, Xianyuan Zhan
arXiv:2602. 05999v3 Announce Type: replace Abstract: How does the amount of compute available to a reinforcement learning (RL) policy affect its learning?
By Raj Ghugare, Micha{\l} Bortkiewicz, Alicja Ziarko, Benjamin Eysenbach
The paper investigates reinforcement learning in constrained partially observable Markov decision processes (CPOMDPs), where an agent must maximize reward while respecting a cost constraint using incomplete state information. By approximating the CPOMDP with finite-memory models and treating observation-action windows as states of a constrained MDP, the authors develop a primal–dual algorithm that offers the first non-asymptotic guarantees on both average reward suboptimality and constraint violation. Under a filter‑stability assumption, they show that the approximation errors decay geometrically with the window length, and validate their method with experiments in partially observable gridworlds.
By Valentina Zaccaria, Philip Jordan, Tingting Ni, Gian Antonio Susto, Maryam Kamgarpour
arXiv:2410.14606v3 Announce Type: replace
Abstract: Learning from a stream of experience as it arrives, also known as streaming learning, is a core part of natural learning. However, reliable streami...
By Mohamed Elsayed, Elena Sorina Lupu, Gautham Vasan, A. Rupam Mahmood
arXiv:2608.30640v1 Announce Type: new
Abstract: While self-supervised approaches to reinforcement learning have achieved strong results by learning representations of states and actions, a key open q...
By Michal Korniak, Kamil Dybek, Benjamin Eysenbach, Marco Bagatella, Micha{\l} Bortkiewicz