arXiv AI
Jun 9
Emergent alignment and the projectability of ethical personas
arXiv:2606. 09475v1 Announce Type: new Abstract: Work on `emergent misalignment' shows that finetuning LLMs on narrow tasks can induce broadly misaligned behavior.
Emergent Alignment
arXiv:2606. 19527v1 Announce Type: new Abstract: Can Large Language Models (LLMs) discern when their own outputs are misaligned with human ethics?
Related stories
arXiv Machine Learning
Jun 5
In-Training Defenses against Emergent Misalignment in Language Models
arXiv:2508. 06249v3 Announce Type: replace Abstract: Fine-tuning lets practitioners repurpose aligned large language models (LLMs) for new domains, yet recent work reveals emergent misalignment (EM): Even a small, domain-specific fine-tune can induce harmful behaviors far outside the target domain.
arXiv AI
Jun 12
LLMs Can Better Capture Human Judgments--With the Right Prompts
arXiv:2606. 12754v1 Announce Type: cross Abstract: Are large language models (LLMs) bad at capturing human judgment?
arXiv AI
Jul 23
Matching Ranks Over Probability Yields Truly Deep Safety Alignment
arXiv:2512. 05518v2 Announce Type: replace-cross Abstract: Open-source Large Language Models (LLMs) play a critical role in the democratization of AI, yet their "open" nature introduces more avenues for malicious actors to misuse them for harmful purposes.
arXiv AI
Jun 2
Safety Alignment of LMs via Non-cooperative Games
arXiv:2512. 20806v3 Announce Type: replace Abstract: Ensuring the safety of language models (LMs) while maintaining their usefulness remains a critical challenge in AI alignment.
arXiv Machine Learning
Jul 13
Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment
arXiv:2601. 22313v2 Announce Type: replace Abstract: Large Language Models (LLMs) are rarely static and are frequently updated in practice.
arXiv AI
1d ago
Inference-Time Mitigation of Adversarial Political Bias in Large Language Models
arXiv:2608. 14629v1 Announce Type: cross Abstract: As Large Language Models (LLMs) become the mainstay for information retrieval and summarization tasks, ensuring that they are always non-partisan and invulnerable to political bias is a critical step towards safer and more trustworthy Artificial Intelligence (AI).
Hugging Face Trending Papers
Jun 17
The Wrong Kind of Right: Quantifying and Localizing Misfired Alignment in LLMs
Warning: This paper studies stereotypes and biases, and contains potentially disturbing examples, used for illustration purposes only. Our findings should not be interpreted as an argument against alignment.
arXiv AI
Jun 3
Consistency Training Can Entrench Misalignment
arXiv:2606. 03810v1 Announce Type: cross Abstract: Consistency training encourages a model to produce similar outputs across related inputs or sampling procedures.
arXiv AI
Jul 3
Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence
arXiv:2604. 08169v2 Announce Type: replace Abstract: Alignment in LLMs is more brittle than commonly assumed: misalignment can be induced by adversarial prompts, benign fine-tuning, emergent misalignment, and goal misgeneralization.
Hugging Face Trending Papers
Jun 2
Consistency Training Can Entrench Misalignment
Consistency training encourages a model to produce similar outputs across related inputs or sampling procedures. Such methods are simple, scalable, and largely label-free, but their effects on model alignment remain poorly understood.
arXiv Machine Learning
Jun 30
The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives
arXiv:2510. 06096v3 Announce Type: replace Abstract: The objectives that Large Language Models (LLMs) implicitly optimize remain dangerously opaque, making trustworthy alignment and auditing a grand challenge.