arXiv AI
Jun 10
SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration
arXiv:2606. 10228v1 Announce Type: cross Abstract: Safe exploration is a prerequisite for deploying reinforcement learning (RL) agents in safety-critical domains.
Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models
arXiv:2607. 13172v1 Announce Type: new Abstract: We address the problem of safely training an agent policy and deploying a good and safe policy, in settings where the environment dynamics are unknown and no suitable reward function is available.
Related stories
arXiv AI
Aug 12
Threat-guided Policy-aware Scene Perturbation for Safe Autonomous Driving with Online Reinforcement Learning
arXiv:2608. 10403v1 Announce Type: new Abstract: Reinforcement learning (RL) has shown promising performance in autonomous driving, yet ensuring the safety of online RL policies remains challenging due to insufficient exposure to safety-critical driving scenes.
arXiv AI
Jun 16
Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds
arXiv:2606. 15385v1 Announce Type: new Abstract: Reward hacking, where AI systems exploit misspecified objectives to achieve high reward without satisfying intended goals, remains a central challenge in AI safety.
arXiv Machine Learning
Jun 18
Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agents
arXiv:2606. 18537v1 Announce Type: new Abstract: Humans often acquire new skills by observing others, since observed behaviors implicitly reveal how to act in an environment.
arXiv AI
Jul 17
RENEW: Towards Learning World Models and Repairing Model Exploitation from Preferences
arXiv:2607. 14180v1 Announce Type: cross Abstract: World models are widely used in offline reinforcement learning (RL) to improve sample efficiency and generate experience beyond a fixed dataset.
arXiv Machine Learning
Jul 27
Safe In-Context Reinforcement Learning
arXiv:2509. 25582v4 Announce Type: replace Abstract: In-context reinforcement learning (ICRL) is an emerging RL paradigm where an agent, after pretraining, can adapt to out-of-distribution test tasks without any parameter updates, instead relying on an expanding context of interaction history.
arXiv Machine Learning
Jul 1
Addressing Over-Refusal in LLMs with Competing Rewards
arXiv:2606. 31748v1 Announce Type: new Abstract: Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones.
arXiv Machine Learning
Jun 30
Theoretical Foundations and Effective Algorithms for Policy-Aware Simulator Learning
arXiv:2605. 29032v2 Announce Type: replace Abstract: Model-based reinforcement learning (MBRL) agents typically learn world models by minimizing predictive loss.
arXiv Machine Learning
Jun 16
Evolutionary Bilevel Reward Shaping for Generalization in Reinforcement Learning
arXiv:2606. 16236v1 Announce Type: new Abstract: Reinforcement learning (RL) often suffers from performance degradation when deployed in environments that differ from those encountered during training.
arXiv AI
Jul 10
Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies
arXiv:2603. 15136v2 Announce Type: replace-cross Abstract: Offline safe reinforcement learning (RL) seeks reward-maximizing policies from static datasets under strict safety constraints.
arXiv AI
Jun 16
Safe Exploration via Policy Priors
arXiv:2601. 19612v3 Announce Type: replace-cross Abstract: Safe exploration is a key requirement for reinforcement learning (RL) agents to learn and adapt online, beyond controlled (e.
arXiv AI
2d ago
ARC: Fair Relative Advantage Comparison in Open-Ended Real-World Interaction
arXiv:2608. 13622v1 Announce Type: new Abstract: Open-ended real-world interaction admits multiple valid behaviors: an agent may answer directly, ask for clarification, provide progress updates, or confirm before acting.