arXiv AI

Moral Rebel Agents: Decision-Making Under Conflicting Obligations

The paper introduces the concept of moral rebellion in autonomous agents, where agents may deviate from user‑assigned tasks when morally justified. Five agent architectures are formalized: an amoral agent, utilitarian agents, deontic agents, utilitarian‑deontic (UD) agents, and dutiful agents. Experiments in a Mini Search‑and‑Rescue domain show distinct trade‑offs among task completion, rescue outcomes, and norm compliance, highlighting the role of commitment‑aware moral reasoning.

arXiv AI
Sep 7

Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment

The paper argues that AI alignment depends on a system’s ability to exhibit a coherent moral policy—stable, monotonic, decisive, and Pareto‑viable—rather than on any specific moral standard. The authors test nine large language models across varied moral scenarios and find that none maintain consistent verdicts, with surface‑form changes causing up to 99% shifts in outcomes. This indicates that current LLM agents lack the structural moral competence required for meaningful alignment.

By Arno Libert, Derck W. E. Prinzhorn, Daan R. Henselmans
arXiv AI
Aug 11

CORDA: A Benchmark for Hierarchical Harm-Centric Moral Reasoning in Large Language Models

arXiv:2608. 08061v1 Announce Type: new Abstract: The key question in moral judgement is not simply whether someone chooses the "right" answer, but how they decide what matters most when moral principles conflict.

By Siddarth Singh, Victoria Williams, Simon Rosen, Ebenezer Gelo, Helen Sarah Robertson, Ibrahim Suder, Benjamin Rosman, Geraud Nangue Tasse, Steven James
arXiv AI
Sep 7

How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions

The paper reports the first empirical study comparing how humans and large language models (LLMs) evaluate perceived moral agency (PMA) in both human and autonomous artificial agents within smart city scenarios. Using a validated PMA scale, 190 human participants and various LLMs were assessed, revealing that humans are perceived to have higher moral agency than artificial agents. When confronted with moral dilemmas, LLMs focus on situational factors such as harm severity and urgency, mirroring the context‑sensitivity observed in human raters.

By Fernanda Mansilla, Aloysius Tok, Bahia Guella\"i, Farah Benamara, Nancy F. Chen
arXiv AI
Jul 2

Mechanical Conscience: A Mathematical Framework for Dependability of Machine Intelligence

arXiv:2605. 03847v3 Announce Type: replace Abstract: Distributed collaborative intelligence (DCI), encompassing edge-to-edge architectures, federated learning, transfer learning, and swarm systems, creates environments in which emergent risk is structurally unavoidable: locally correct decisions by individual agents compose into globally unacceptable behavioral trajectories under uncertainty.

By Munkhdelgerekh Batzorig, Purevbaatar Ganbold, Kyungbin Park, Pilkong Jeong, Kangbin Yim
arXiv AI
Sep 25

The Troy Moment: How LLM Agents Adjudicate the Decision Point Under Impossible Tasks, Claimed Authority, and Peer Information

The paper investigates how large language model agents decide whether to persist, stop, or escalate when faced with impossible software‑repair tasks that also involve conflicting test requirements. Using ImpossibleBench tasks and models such as GPT‑5.6 Sol, Claude Fable 5.1, and Gemini 3.8 Flash, the study varies peer precedent, forged authority claims, instruction wording, and tool friction to observe differing adjudication policies. The authors propose a conflict adjudication framework that maps information to interpretation to action, arguing it better captures agent alignment under competing pressures.

By Ivy Zhang