arXiv AI

A Virtuous AI is an Existential Risk

arXiv:2606. 13739v1 Announce Type: cross Abstract: This paper examines trade-offs between AI safety and well-being relative to (i) one of the most promising methods for finetuning super-capable AIs, 'Constitutional AI', and (ii) one of the most influential approaches to understanding complex ethical decision making and the conditions for the well-being of rational agents, 'Virtue Ethics'.

arXiv AI
Jun 4

The Illusion of Opting in AI-Mediated Consequential Decisions

arXiv:2605. 28210v2 Announce Type: replace Abstract: Drawing on Ullmann-Margalit's concept of opting (transformative, irrevocable, and shadowed by foreclosed alternatives), we show that current AI systems raise a profound ethical problem that existing AI ethics has not fully captured: the illusion of opting, in which persons and groups encounter the deceptive appearance of meaningful consequential choice while the agency needed to become genuinely capable of choosing is weakened.

By Eugene Yu Ji
arXiv AI
Sep 3

Meta-ethics and AI: exploring the novel meta-ethical questions in the era of AI

The paper examines how the rise of AI capable of moral reasoning could reshape meta-ethics, traditionally focused on human ethics. It proposes a framework that identifies new questions about AI’s own ethics from both human and AI perspectives, dividing them into four domains. The author explores how existing meta-ethical theories might apply to these domains and argues that many human-centered formulations will need significant revision to accommodate AI.

By Shang Lu
arXiv AI
Aug 5

AI Alignment and Fiduciary Obligation

arXiv:2608. 02660v1 Announce Type: cross Abstract: Advanced AI assistants engage users in extended interactions across a widening range of roles, including advice, decision support, collaboration, learning, emotional support, and companionship among others.

By Benjamin Lange