arXiv AI
Aug 12
Toward a Theory of Value in AI Alignment
arXiv:2608. 10327v1 Announce Type: new Abstract: Can AI systems be aligned to human values?
Fragility of Value under Imperfect Alignment
arXiv:2607. 28881v1 Announce Type: new Abstract: As more responsibility is placed upon AI systems, it becomes increasingly important to guarantee that these systems are aligned with humanity.
Related stories
arXiv Machine Learning
Jun 2
Safety Game: Inference-Time Alignment of Black-Box LLMs via Constrained Optimization
arXiv:2510. 09330v3 Announce Type: replace Abstract: Ensuring that large language models (LLMs) comply with safety requirements is a central challenge in AI deployment.
arXiv AI
Jun 2
SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
arXiv:2606. 02530v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) with human values often degrades their general capabilities, termed the alignment tax.
arXiv AI
Jul 2
Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction
arXiv:2607. 00001v1 Announce Type: new Abstract: Most approaches to AI alignment treat human preferences as fixed targets to be inferred and optimized.
Hugging Face Trending Papers
Jul 8
Alignment Plausibility: A New Standard for Assuring AI in Healthcare
Large language models (LLMs) have become significant providers of mental health support, yet they remain products of an attention economy whose operational and commercial targets favour sustained engagement over the friction that effective psychological support often requires. Developers' safety responses have been largely reactive, addressing the most visible and acute harms while subtler, longer-term patterns of risk (e.
arXiv AI
Jun 12
Strategic Decision Support for AI Agents
arXiv:2606. 12587v1 Announce Type: new Abstract: Traditionally, decision support studies how humans use machine learning models to make better decisions.
OpenAI Blog
Aug 24, 2022
Our approach to alignment research
We are improving our AI systems’ ability to learn from human feedback and to assist humans at evaluating AI. Our goal is to build a sufficiently aligned AI system that can help us solve all other alignment problems.
arXiv Machine Learning
Jun 4
Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
arXiv:2603. 07445v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often require fine-tuning (FT) to perform well on downstream tasks, but FT can induce safety-alignment drift even when the training dataset contains only benign data.
arXiv AI
Jul 10
Alignment Plausibility: A New Standard for Assuring AI in Healthcare
arXiv:2607. 07766v1 Announce Type: new Abstract: Large language models (LLMs) have become significant providers of mental health support, yet they remain products of an attention economy whose operational and commercial targets favour sustained engagement over the friction that effective psychological support often requires.
arXiv Machine Learning
Jun 5
Alignment Risks from Capability-Seeking RL Training
arXiv:2602. 12124v2 Announce Type: replace Abstract: While most AI alignment research focuses on preventing models from generating explicitly harmful content, a more subtle risk arises from capability-seeking RL training in vulnerable environments.
arXiv AI
Jul 17
Align AI to Dynamic Human-AI Workflows
arXiv:2607. 14240v1 Announce Type: new Abstract: Current alignment approaches typically focus on emulating human behavior using static representations of human preferences, failing to capture the dynamic, context-dependent nature of real-world human-AI interactions.
arXiv AI
Jun 29
The Alignment Target Problem: Divergent Moral Judgments of Humans, AI Systems, and Their Designers
arXiv:2604. 24155v3 Announce Type: replace-cross Abstract: The project of aligning machine behavior with human values raises a basic problem: whose moral expectations should guide AI decision-making?