arXiv AI
Mechanism Design for Alignment and Control
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 2
Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction
arXiv:2607. 00001v1 Announce Type: new Abstract: Most approaches to AI alignment treat human preferences as fixed targets to be inferred and optimized.
arXiv AI
Sep 15
Delegating Authorization to Misaligned Agents: Coalitional Alignment and Safe Control
The paper studies how to safely delegate action approval to multiple AI reviewers when the reviewers themselves may be misaligned. It introduces a weaker condition—k‑robust coalitional alignment—under which a threshold rule that tolerates up to k disapprovals guarantees that the principal’s expected utility is at least as good as a baseline policy. The authors extend this characterization to sequential decision‑making in discounted MDPs and show that full‑panel coverage of reward functions ensures safety in Nash equilibria, while more permissive thresholds can lead to unsafe outcomes. Experiments demonstrate that collective review can remain sound even when individual reviewers are not fully aligned, provided some disapprovals are allowed.
arXiv AI
Jul 14
Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems
arXiv:2607. 09766v1 Announce Type: new Abstract: AI agents are increasingly deployed in shared environments where they pursue diverse goals and compete for rewards.
arXiv AI
Jun 9
Payoff scaling shapes cooperation in LLM agents across languages
arXiv:2601. 19082v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents that negotiate, coordinate, and act on behalf of users.
arXiv AI
Sep 12
Autonomy, Social Norms, and Alignment: Towards a Developmental Framework for Autonomous Artificial Agents
The paper proposes a developmental framework for autonomous artificial agents that emphasizes learning social norms and alignment through direct interaction with dynamic environments. It argues that intrinsic motivations such as curiosity and competence can guide exploration, but also complicate alignment with human goals. By drawing parallels to child development, the authors suggest that regulatory sandboxes serve as pedagogical spaces where agents gradually acquire moral agency and adapt their behaviors through experience and cooperation.
arXiv AI
Jun 3
Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AI
arXiv:2605. 08426v2 Announce Type: replace-cross Abstract: Ensuring that AI agents behave safely and beneficially when interacting with other parties has emerged as one of the central challenges of modern AI safety.