arXiv Machine Learning
Jun 16
On the Role of Computation in Reinforcement Learning
arXiv:2602. 05999v3 Announce Type: replace Abstract: How does the amount of compute available to a reinforcement learning (RL) policy affect its learning?
From Causal Graphs to Markovian States: Multi-Order State Exposure for Deep RL
Related stories
arXiv Machine Learning
Oct 2
Reinforcement Learning with Complex (valued) Memories
arXiv:2609.38598v1 Announce Type: new Abstract: Partially observable environments pose a fundamental challenge in deep reinforcement learning, requiring agents to compress temporal information from o...
arXiv Machine Learning
Jul 28
Hierarchical Reinforcement Learning with Optimal Level Synchronization Based on Flow-Based Deep Generative Model
arXiv:2107. 08183v2 Announce Type: replace Abstract: High-dimensional state and action spaces combined with sparse reward structures in reinforcement learning (RL) environments typically require advanced control architectures.
arXiv AI
Jun 30
TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning
arXiv:2401. 11512v2 Announce Type: replace-cross Abstract: Identifying the most suitable variables to represent the state is a fundamental challenge in Reinforcement Learning (RL).
arXiv Machine Learning
Sep 10
Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions
The paper introduces state abstractions that preserve the difference of Q‑functions for offline reinforcement learning, aiming to exclude irrelevant dynamics from rich state data. It proposes a dynamic generalization of the R‑learner that uses orthogonal estimation and sparse learning to estimate the Q‑function contrast, achieving faster convergence and consistency under a margin condition. Experiments on simulated and simulator‑augmented real data show variance reductions and demonstrate that the necessary information for sequential decision‑making can be smaller than that required for full state prediction.
arXiv AI
Jul 7
Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learning
arXiv:2605. 26012v2 Announce Type: replace-cross Abstract: Deep reinforcement learning (RL) agents commonly rely on high-dimensional neural representations, despite growing evidence that task-relevant value and policy structure may be intrinsically low-dimensional.
arXiv Machine Learning
Jul 21
Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
arXiv:2509. 02522v3 Announce Type: replace-cross Abstract: Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have empowered large language models (LLMs) to tackle challenging reasoning tasks such as mathematics and programming, however existing RLVR methods often suffer from sparse reward signals and unstable policy gradient updates inherent to RL-based approaches.
arXiv Machine Learning
Aug 11
V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control
arXiv:2608. 07870v1 Announce Type: new Abstract: Improving sample efficiency remains a core challenge in reinforcement learning (RL), especially in real-world settings like robotics, where data collection is costly.
arXiv Machine Learning
Jun 5
Representation Learning Enables Scalable Multitask Deep Reinforcement Learning
arXiv:2606. 05555v1 Announce Type: new Abstract: Scaling reinforcement learning (RL) to diverse multitask settings remains a central challenge.
arXiv AI
Oct 15
Growing and Transferring Pathways for LLM Reinforcement Learning
arXiv:2610.16489v1 Announce Type: cross Abstract: Reinforcement learning (RL) can substantially improve the reasoning capabilities of large language models, but an effective training run requires con...
arXiv Machine Learning
Aug 24
Decoupling Policy Extraction for Offline Reinforcement Learning
arXiv:2608.20909v1 Announce Type: new Abstract: Offline RL methods commonly jointly train the actor and critic, where the critic is used to guide the actor toward higher-value actions. This coupled l...
arXiv AI
Oct 12
Does Learning Recurrent Networks Matter in Partially Observable RL?
The paper investigates whether a frozen recurrent network can provide sufficient memory for partially observable Markov decision processes (POMDPs). Using a fixed recurrent network combined with a trainable output module, the authors demonstrate that such networks can recall information over hundreds of steps and achieve competitive control performance across diverse POMDP tasks, while using fewer than half the trainable parameters of fully trainable recurrent networks. These findings suggest that learning the recurrent memory module is not always necessary, offering a simple, parameter‑efficient baseline for partially observable reinforcement learning.