The paper investigates how large language models encode moral knowledge by training linear probes for each Moral Foundations Theory category and analyzing their geometric relationships. It finds that the model’s moral directions are largely independent yet share a common component, indicating integration rather than collapse into a single detector. This structure is consistent across architectures, emerges early in pre‑training, and reflects corpus statistics rather than the individualizing/binding distinction of Moral Foundations Theory.
The paper argues that AI alignment depends on a system’s ability to exhibit a coherent moral policy—stable, monotonic, decisive, and Pareto‑viable—rather than on any specific moral standard. The authors test nine large language models across varied moral scenarios and find that none maintain consistent verdicts, with surface‑form changes causing up to 99% shifts in outcomes. This indicates that current LLM agents lack the structural moral competence required for meaningful alignment.
By Arno Libert, Derck W. E. Prinzhorn, Daan R. Henselmans
The study investigates how large language models encode moral knowledge by training linear probes for each category of Moral Foundations Theory. It finds that the model’s representations for different moral foundations occupy distinct, largely independent dimensions yet share a common positive component, indicating an integrated but nuanced moral structure. This geometry is consistent across model architectures and scales, emerges early in pre‑training, and reflects corpus statistics rather than the individualizing/binding distinction of the theory.
By Orion Reblitz-Richardson
arXiv:2606. 06972v1 Announce Type: new Abstract: Ensuring that agent behaviours are aligned with human moral values inevitably raises the problem of how to account for the plurality of moral perspectives that societies -- and even individuals -- typically adopt.
By Jazon Szabo, Sanjay Modgil
arXiv:2608. 12368v1 Announce Type: new Abstract: Agreement with human judgments is a common proxy for evaluating the alignment of large language models (LLMs).
By Octavian M. Machidon, Alina L. Machidon, Vojko Strahovnik, Mateja Centa Strahovnik, Jonas Miklav\v{c}i\v{c}, Marko Robnik \v{S}ikonja
arXiv:2608.28610v1 Announce Type: new
Abstract: Existing LLM moral evaluations typically present models with isolated moral vignettes and elicit a single-shot decision, neglecting a factor known to p...
By Fangyuan Zhang, Dong Yu, Pengyuan Liu
arXiv:2603.23114v2 Announce Type: replace
Abstract: A human's moral decision depends heavily on the context. Yet research on LLM morality has largely studied fixed scenarios. We address this gap by i...
By Adrian Sauter, Mona Schirmer
arXiv:2608. 08061v1 Announce Type: new Abstract: The key question in moral judgement is not simply whether someone chooses the "right" answer, but how they decide what matters most when moral principles conflict.
By Siddarth Singh, Victoria Williams, Simon Rosen, Ebenezer Gelo, Helen Sarah Robertson, Ibrahim Suder, Benjamin Rosman, Geraud Nangue Tasse, Steven James
arXiv:2609.21992v1 Announce Type: new
Abstract: Most work in computational ethics treats annotator disagreement on moral content as noise to be voted away, collapsed into majority vote or the more pe...
By Maciej Skorski
arXiv:2606. 11635v1 Announce Type: cross Abstract: For highly capable AI systems to operate safely in dynamic, open-ended environments, they must be able to identify, understand, and respond to moral reasons for action, and constrain their behaviour accordingly.
By Menghang Zhu, Seth Lazar
arXiv:2608. 14522v1 Announce Type: new Abstract: As AI systems make more morally loaded decisions across society, one response has been moral preference elicitation.
By Taenyun Kim, Edyta Bogucka, Daniele Quercia
The study evaluates how large language models (LLMs) respond to the Norwegian Moral Foundations Questionnaire (MFQ‑30) and compares their moral profiles to a Norwegian human sample. Six open‑weight LLMs were tested, with half engaging with the questionnaire and the other half producing flat, near‑human responses. Two steering methods—prompt‑level persona steering and activation‑level ActAdd—were applied; a neutral Nordic persona improved alignment with the Norwegian mean by 44‑77%, while ActAdd flattened profiles without targeting specific foundations.
By Hans Andersen, David Dichas