The paper introduces memoroids, a monoid‑based framework that reformulates memory models like RNNs and Transformers for better handling of long sequences in reinforcement learning. It critiques traditional batching approaches in recurrent RL, identifies their shortcomings, and proposes a new batching method using memoroids that enhances sample efficiency, boosts return, and simplifies recurrent loss implementation.
arXiv:2604. 01577v3 Announce Type: replace-cross Abstract: We study out of distribution generalization in streaming tasks where models are trained on short sequences but must operate over much longer, unknown horizons under bounded memory.
The paper examines whether a frozen recurrent network can supply sufficient memory for partially observable reinforcement learning tasks, instead of training the recurrent module jointly with the policy. Using a fixed recurrent network paired with a trainable output layer, the authors test this setup on various POMDPs with different memory demands, observation types, and action spaces. Results show that frozen networks can recall information over hundreds of steps and achieve competitive performance while using fewer than half the trainable parameters of fully trainable recurrent models.
The paper investigates whether a frozen recurrent network can provide sufficient memory for partially observable Markov decision processes (POMDPs). Using a fixed recurrent network combined with a trainable output module, the authors demonstrate that such networks can recall information over hundreds of steps and achieve competitive control performance across diverse POMDP tasks, while using fewer than half the trainable parameters of fully trainable recurrent networks. These findings suggest that learning the recurrent memory module is not always necessary, offering a simple, parameter‑efficient baseline for partially observable reinforcement learning.
arXiv:2107. 08183v2 Announce Type: replace Abstract: High-dimensional state and action spaces combined with sparse reward structures in reinforcement learning (RL) environments typically require advanced control architectures.
arXiv:2606. 10346v1 Announce Type: new Abstract: Reinforcement learning has become a key paradigm for eliciting reasoning abilities in large language models, where exploration is crucial for discovering effective solution trajectories.
arXiv:2608. 06595v1 Announce Type: cross Abstract: Neural networks applied to sequential decision-making tasks typically rely on latent representations of environment states.
arXiv:2605. 25638v2 Announce Type: replace-cross Abstract: Policy loss estimation remains a fundamental and long-standing challenge in reinforcement learning (RL) for diffusion language models (DLMs).
The paper introduces Round-Trip Reinforcement Learning (RTRL), a framework that trains chemical language models to improve round‑trip consistency by rewarding successful forward and reverse transformations. By iteratively training forward and reverse mappings, RTRL leverages abundant unlabeled chemical data to enhance both consistency and overall performance across supervised, self‑supervised, and synthetic data regimes. Experiments show that RTRL outperforms strong baselines, demonstrating that round‑trip consistency can be treated as a trainable objective for more robust foundation models.
arXiv:2610.29671v1 Announce Type: new Abstract: Long-term agent memory is shifting from predefined workflows toward learnable policies for memory construction and retrieval. Existing learning-based m...
arXiv:2602. 14169v2 Announce Type: replace-cross Abstract: Effective exploration is a key challenge in reinforcement learning for large language models: discovering high-quality trajectories within a limited sampling budget from the vast natural language sequence space.
arXiv:2607. 04713v1 Announce Type: cross Abstract: Reinforcement learning holds significant potential for training large language models (LLMs) to handle multi-turn interactive tasks.
arXiv:2603. 04910v2 Announce Type: replace-cross Abstract: Imitation learning from human demonstrations has achieved significant success in robotic control, yet most visuomotor policies still condition on single-step observations or short-context histories, making them struggle with non-Markovian tasks that require long-term memory.