arXiv AI
Jul 2
Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction
arXiv:2607. 00001v1 Announce Type: new Abstract: Most approaches to AI alignment treat human preferences as fixed targets to be inferred and optimized.
Mechanism Design for Alignment and Control
Related stories
arXiv AI
Jul 14
Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems
arXiv:2607. 09766v1 Announce Type: new Abstract: AI agents are increasingly deployed in shared environments where they pursue diverse goals and compete for rewards.
arXiv AI
Jun 9
Payoff scaling shapes cooperation in LLM agents across languages
arXiv:2601. 19082v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents that negotiate, coordinate, and act on behalf of users.
arXiv AI
Jun 3
Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AI
arXiv:2605. 08426v2 Announce Type: replace-cross Abstract: Ensuring that AI agents behave safely and beneficially when interacting with other parties has emerged as one of the central challenges of modern AI safety.
arXiv AI
Jul 29
The Possibility of Artificial Intelligence Becoming a Subject and the Alignment Problem
arXiv:2604. 14990v2 Announce Type: replace Abstract: The prospect of Artificial General Intelligence (AGI) is increasingly driving institutional decisions, and alignment of AGI is a hard problem.
arXiv AI
Jul 20
Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation
arXiv:2607. 15434v1 Announce Type: cross Abstract: Multi-agent systems routinely place one AI agent in authority over another.
arXiv AI
Jun 3
Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions
arXiv:2606. 02859v1 Announce Type: cross Abstract: How can a population of agents self-orchestrate and self-adapt into stronger collective intelligence without centralized control?
arXiv AI
Jul 7
Governed Individuation: Cryptographically Decoupling an Agent's Learning from Its Authority
arXiv:2607. 04613v1 Announce Type: new Abstract: Autonomous agents are moving from sandboxed text generators to operators of code, data, and physical infrastructure, and they increasingly learn while deployed.
arXiv AI
Aug 28
Five Primitives for Governing Autonomous AI Agents at Runtime
The paper proposes five runtime primitives—discovery, identity, governance, attestation, and supply chain—to manage autonomous AI agents in enterprise settings. It argues that traditional control models fail because agents are transient, model-driven, and self‑discoverable, making runtime governance essential. The authors detail an implementation that mediates agent actions against policy, authorizes them via a per‑tenant vocabulary, and records them in a verifiable ledger, noting the associated operational costs and partial deployment status.
arXiv AI
Jul 2
A Contextual-Bandit Oversight Game with Two-Sided Informational Asymmetry
arXiv:2607. 00155v1 Announce Type: new Abstract: We study runtime human oversight of an AI agent when private information runs in both directions: the human privately knows her reward function, while the AI privately knows the quality of the action it proposes.
arXiv AI
Jul 7
CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas
arXiv:2604. 15267v2 Announce Type: replace-cross Abstract: It is increasingly important that LLM agents interact effectively and safely with other goal-pursuing agents, yet, recent works report the opposite trend: LLMs with stronger reasoning capabilities behave _less_ cooperatively in mixed-motive games such as the prisoner's dilemma and public goods settings.
arXiv AI
Aug 11
The Theory of Strategic Evolution: Games with Endogenous Players and the Seven Laws of Strategic Replicators
arXiv:2512. 07901v4 Announce Type: replace-cross Abstract: Von Neumann founded both game theory and the theory of self-reproducing automata, but the two programs never merged.