arXiv AI
Jul 13
Towards Shutdownable Agents: Generalizing Stochastic Choice in RL Agents and LLMs
arXiv:2604. 17502v4 Announce Type: replace Abstract: Misaligned artificial agents might resist shutdown.
Shutdownable Agents through POST-Agency
arXiv:2505. 20203v4 Announce Type: replace Abstract: Many fear that future artificial agents will resist shutdown.
Related stories
arXiv AI
Jun 9
Revisiting the shutdown problem
arXiv:2606. 08296v1 Announce Type: new Abstract: A key premise in leading arguments for existential risk from artificial intelligence is that malfunctioning artificial agents could not be easily shut down.
arXiv Machine Learning
2d ago
What preferences can - and cannot - predict in multi-agent online learning
arXiv:2608. 13810v1 Announce Type: cross Abstract: We examine the interplay between ordinal, preference-based solution concepts in games and the long-run behavior of game dynamics, asking in particular to what extent the combinatorial data of a game -- its preference graph -- determine the outcomes of no-regret learning dynamics -- such as follow-the-regularized-leader (FTRL).
arXiv AI
Jul 23
Will the Agent Recuse, and Will It Stop? Measuring LLM-Agent Compliance with In-Band Governance Signals at the Access Door and Mid-Flight
arXiv:2606. 06460v3 Announce Type: replace-cross Abstract: Autonomous LLM agents increasingly hold real credentials and operate infrastructure with no human in the loop, yet operators have no standard way to tell an agent a resource is off-limits, or to ask a running agent to stand down: access controls either admit it or hard-fail it.
arXiv AI
Jun 16
Resilient Consensus in Agentic AI
arXiv:2606. 15024v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly deployed in multi-agent systems where they must coordinate and agree on shared decisions.
arXiv Machine Learning
Aug 4
Training Small LLMs as Spatial Multi-Agent Policies
arXiv:2608. 01425v1 Announce Type: cross Abstract: Training LLM-based multi-agent systems with multi-agent reinforcement learning is rapidly gaining traction, and a parallel line of work argues that such systems should be judged by their behavior, not only their reward.
arXiv AI
Jun 3
Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AI
arXiv:2605. 08426v2 Announce Type: replace-cross Abstract: Ensuring that AI agents behave safely and beneficially when interacting with other parties has emerged as one of the central challenges of modern AI safety.
arXiv AI
Jul 14
Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems
arXiv:2607. 09766v1 Announce Type: new Abstract: AI agents are increasingly deployed in shared environments where they pursue diverse goals and compete for rewards.
arXiv AI
Jul 23
JANUS: Foreseeing Latent Risk for Long-Horizon Agent Safety
arXiv:2607. 19913v1 Announce Type: new Abstract: Agent safety is moving from content moderation toward preventing operational failures before tool-using agents act.
arXiv AI
Jun 2
Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults
arXiv:2606. 00914v1 Announce Type: new Abstract: LLM agents increasingly act after consuming ranked external information streams such as social feeds, search results, retrieval contexts, and email queues, yet safety evaluations almost always test the model or the user prompt in isolation, never the upstream ranker that decides what the agent reads just before it acts.
arXiv AI
Jun 11
Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RL
arXiv:2307. 01472v2 Announce Type: replace Abstract: We present a novel Diffusion Offline Multi-agent Model (DOM2) for offline Multi-Agent Reinforcement Learning (MARL).
Hugging Face Trending Papers
Jul 22
JANUS: Foreseeing Latent Risk for Long-Horizon Agent Safety
Agent safety is moving from content moderation toward preventing operational failures before tool-using agents act. We propose Janus, a foresight-oriented framework for long-horizon agent safety that trains guards to anticipate delayed risks from partial trajectories.