arXiv AI
Jul 2
Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction
arXiv:2607. 00001v1 Announce Type: new Abstract: Most approaches to AI alignment treat human preferences as fixed targets to be inferred and optimized.
Mechanism Design for Alignment and Control
Related stories
arXiv AI
Sep 15
Delegating Authorization to Misaligned Agents: Coalitional Alignment and Safe Control
The paper studies how to safely delegate action approval to multiple AI reviewers when the reviewers themselves may be misaligned. It introduces a weaker condition—k‑robust coalitional alignment—under which a threshold rule that tolerates up to k disapprovals guarantees that the principal’s expected utility is at least as good as a baseline policy. The authors extend this characterization to sequential decision‑making in discounted MDPs and show that full‑panel coverage of reward functions ensures safety in Nash equilibria, while more permissive thresholds can lead to unsafe outcomes. Experiments demonstrate that collective review can remain sound even when individual reviewers are not fully aligned, provided some disapprovals are allowed.
arXiv AI
Jul 14
Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems
arXiv:2607. 09766v1 Announce Type: new Abstract: AI agents are increasingly deployed in shared environments where they pursue diverse goals and compete for rewards.
arXiv AI
Jun 9
Payoff scaling shapes cooperation in LLM agents across languages
arXiv:2601. 19082v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents that negotiate, coordinate, and act on behalf of users.
arXiv AI
Sep 12
Autonomy, Social Norms, and Alignment: Towards a Developmental Framework for Autonomous Artificial Agents
The paper proposes a developmental framework for autonomous artificial agents that emphasizes learning social norms and alignment through direct interaction with dynamic environments. It argues that intrinsic motivations such as curiosity and competence can guide exploration, but also complicate alignment with human goals. By drawing parallels to child development, the authors suggest that regulatory sandboxes serve as pedagogical spaces where agents gradually acquire moral agency and adapt their behaviors through experience and cooperation.
arXiv AI
Jun 3
Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AI
arXiv:2605. 08426v2 Announce Type: replace-cross Abstract: Ensuring that AI agents behave safely and beneficially when interacting with other parties has emerged as one of the central challenges of modern AI safety.
arXiv AI
Jul 29
The Possibility of Artificial Intelligence Becoming a Subject and the Alignment Problem
arXiv:2604. 14990v2 Announce Type: replace Abstract: The prospect of Artificial General Intelligence (AGI) is increasingly driving institutional decisions, and alignment of AGI is a hard problem.
arXiv AI
Sep 10
Agentic Pressure: The Endogenous Entropy of Reliable Autonomy
arXiv:2609.05995v1 Announce Type: new Abstract: Achieving reliable autonomy in the wild requires agents to sustain continuous operations across long-horizon trajectories. However, as agents navigate...
arXiv AI
Jul 20
Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation
arXiv:2607. 15434v1 Announce Type: cross Abstract: Multi-agent systems routinely place one AI agent in authority over another.
arXiv AI
Jun 3
Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions
arXiv:2606. 02859v1 Announce Type: cross Abstract: How can a population of agents self-orchestrate and self-adapt into stronger collective intelligence without centralized control?
arXiv AI
Oct 15
Computational Reflective Equilibrium for Agentic AI: A Rawlsian Alignment Scaffold for Governance
arXiv:2610.16015v1 Announce Type: new Abstract: The rapid deployment of agentic AI has introduced complex challenges for the ethics and governance of AI systems. Agents built on large language models...
arXiv AI
Jul 7
Governed Individuation: Cryptographically Decoupling an Agent's Learning from Its Authority
arXiv:2607. 04613v1 Announce Type: new Abstract: Autonomous agents are moving from sandboxed text generators to operators of code, data, and physical infrastructure, and they increasingly learn while deployed.