arXiv AI
Aug 11
A Unified Framework for Dynamic Reward Shaping in Reinforcement Learning
arXiv:2608. 08158v1 Announce Type: new Abstract: Sparse, delayed, and weakly informative rewards remain central obstacles to efficient reinforcement learning.
Policy Complexity, Reaction Time, and Bounded Rationality in Reinforcement Learning
The paper introduces MI‑SARSA, an on‑policy temporal‑difference algorithm that incorporates mutual‑information regularization to model bounded rationality in reinforcement learning. By penalizing state‑specific deviations from a learned marginal action prior, the algorithm selectively uses state information only when the expected return outweighs the informational cost, yielding a reward‑complexity tradeoff. MI‑SARSA also predicts reaction times, showing that stronger information penalties lead to simpler policies, lower control costs, and faster responses, while regularization mitigates performance loss after environmental shifts at the expense of asymptotic return.
Related stories
arXiv Machine Learning
Sep 10
Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions
The paper introduces state abstractions that preserve the difference of Q‑functions for offline reinforcement learning, aiming to exclude irrelevant dynamics from rich state data. It proposes a dynamic generalization of the R‑learner that uses orthogonal estimation and sparse learning to estimate the Q‑function contrast, achieving faster convergence and consistency under a margin condition. Experiments on simulated and simulator‑augmented real data show variance reductions and demonstrate that the necessary information for sequential decision‑making can be smaller than that required for full state prediction.
arXiv Machine Learning
Aug 6
Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning
arXiv:2608. 05111v1 Announce Type: new Abstract: In partially observable reinforcement learning, agents face a dual bottleneck: they must explore to encounter rewarding states and retain that experience in memory to optimize their policies.
arXiv Machine Learning
Aug 10
Learning Suffers More Than the Policy Class Under Partial Observability: A Closed-Form Analysis
arXiv:2608. 07228v1 Announce Type: new Abstract: When a reinforcement learning agent cannot observe the full state, we usually blame its policies: it cannot see enough to represent a good one.
arXiv Statistics ML
2d ago
Meta-reinforcement learning with minimum attention
arXiv:2505.16741v5 Announce Type: replace-cross Abstract: Minimum attention applies the least action principle in changes of control concerning state and time, first proposed by Brockett. The involve...
Hugging Face Trending Papers
Sep 23
Limiting-Kernel Q($λ$): Bridging Short and Long Horizons
In value-based reinforcement learning, improving the accuracy of policy evaluation has been shown to improve downstream policy optimization performance. The widely adopted family of approximations rel...
arXiv Machine Learning
Sep 24
Limiting-Kernel Q($\lambda$): Bridging Short and Long Horizons
Limiting‑Kernel Q(λ) (LKQL) is an off‑policy value estimator that blends n‑step truncation with a long‑horizon approximation based on the limiting kernel. It maintains the computational efficiency of n‑step methods while improving policy evaluation accuracy, especially for long‑horizon tasks. The authors prove faster convergence of LKQL’s operator under aperiodicity and near‑on‑policy conditions, and demonstrate empirical gains on MuJoCo continuous‑control benchmarks.
arXiv Machine Learning
Jul 30
Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback
arXiv:2607. 26094v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback (RLHF) is the standard approach for aligning large language models with human preferences, but its quality is limited by static, task-agnostic reward models.
arXiv AI
Jun 30
TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning
arXiv:2401. 11512v2 Announce Type: replace-cross Abstract: Identifying the most suitable variables to represent the state is a fundamental challenge in Reinforcement Learning (RL).
arXiv AI
Jul 24
Adaptive Multi-Horizon Reinforcement Learning
arXiv:2607. 20656v1 Announce Type: cross Abstract: Effective decision-making in complex and changing environments requires balancing short-term and long-term consequences.
arXiv Machine Learning
Aug 12
IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning
arXiv:2608. 10634v1 Announce Type: new Abstract: Model-based reinforcement learning (MBRL), which learns environment dynamics to generate synthetic experience, is a promising approach to sample-efficient decision making.
arXiv AI
Sep 1
BCPPO: Bachelier-Inspired Constrained Proximal Policy Optimization for Tail-Risk-Aware Safe Reinforcement Learning
BCPPO is a new variant of Proximal Policy Optimization that uses Bachelier-inspired cost‑prediction networks to generate a smooth penalty based on disagreement among critics. The method keeps temporal‑difference learning unchanged, applies a saturation‑aware controller to manage cost penalties, and deploys only the policy network. Across extensive experiments, BCPPO outperforms comparators in achieving higher mean returns while maintaining lower or comparable CVaR in all tested tasks.