arXiv AI

Acceptable Misalignment

Hugging Face Trending Papers
Jul 8

Alignment Plausibility: A New Standard for Assuring AI in Healthcare

Large language models (LLMs) have become significant providers of mental health support, yet they remain products of an attention economy whose operational and commercial targets favour sustained engagement over the friction that effective psychological support often requires. Developers' safety responses have been largely reactive, addressing the most visible and acute harms while subtler, longer-term patterns of risk (e.

arXiv AI
Oct 27

From Language to Drives: A Multimodal Framework for Aligning Autonomous Robots with Human Purposes

The paper presents a multimodal framework that aligns open-ended learning autonomous robots with human purposes using large language models (LLMs) and vision-language models (VLMs). It transforms human intentions expressed in natural language into robot missions and drives through a three-layer architecture—semantic alignment, mission generation, and drive modeling—allowing bidirectional interaction. Experiments in simulation and on a physical robot show that the system generates valid missions and behaviors, maintaining alignment with human intent.

By Diana Cerdas-Vargas, Alejandro Romero, Richard J. Duro
arXiv AI
Oct 7

SIGMA: Self-Improving Alignment Generalization from a Model Spec

SIGMA is a self‑improving alignment framework that uses a model’s own reasoning to generate diverse alignment dilemma scenarios from a simple "Model Spec" and then fine‑tunes the model with self‑judged rewards. By performing spec‑guided task synthesis and rubric‑based reinforcement learning, SIGMA improves safety alignment in multi‑turn agentic settings, reducing harmfulness and misalignment scores while preserving general capability. The approach demonstrates that a model can autonomously strengthen its own safety reasoning without external supervision.

By Jingyu Zhang, Shruti Palaskar, Daniel Khashabi, Benjamin Van Durme, Leon A. Gatys, Joseph Yitan Cheng
arXiv AI
Jul 10

Alignment Plausibility: A New Standard for Assuring AI in Healthcare

arXiv:2607. 07766v1 Announce Type: new Abstract: Large language models (LLMs) have become significant providers of mental health support, yet they remain products of an attention economy whose operational and commercial targets favour sustained engagement over the friction that effective psychological support often requires.

By Gwydion Williams, Sara Zannone, Bilal A Mateen
OpenAI Blog
Aug 24, 2022

Our approach to alignment research

We are improving our AI systems’ ability to learn from human feedback and to assist humans at evaluating AI. Our goal is to build a sufficiently aligned AI system that can help us solve all other alignment problems.

arXiv AI
Sep 12

Autonomy, Social Norms, and Alignment: Towards a Developmental Framework for Autonomous Artificial Agents

The paper proposes a developmental framework for autonomous artificial agents that emphasizes learning social norms and alignment through direct interaction with dynamic environments. It argues that intrinsic motivations such as curiosity and competence can guide exploration, but also complicate alignment with human goals. By drawing parallels to child development, the authors suggest that regulatory sandboxes serve as pedagogical spaces where agents gradually acquire moral agency and adapt their behaviors through experience and cooperation.

By Marica Notte, Ludovica Marinucci, Vieri Giuliano Santucci