arXiv Machine Learning June 2, 2026 By Maryam Hashemzadeh, Jerry Huang, Minseon Kim, Marc-Alexandre C\^ot\'e, Sarath Chandar

Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routing

Read the original on arXiv Machine Learning →

arXiv:2606. 00686v1 Announce Type: new Abstract: The prevailing paradigm in large language model (LLM) alignment operates via erasure, filtering unsafe data or training models to strictly refuse harmful prompts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 23

Matching Ranks Over Probability Yields Truly Deep Safety Alignment

arXiv:2512. 05518v2 Announce Type: replace-cross Abstract: Open-source Large Language Models (LLMs) play a critical role in the democratization of AI, yet their "open" nature introduces more avenues for malicious actors to misuse them for harmful purposes.

By Jason Vega, Gagandeep Singh

llms fine-tuning safety

arXiv AI

Jun 18

Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection

arXiv:2606. 19168v1 Announce Type: new Abstract: To achieve deeper safety alignment for large language models (LLMs), recent efforts have studied how to push safety interventions earlier into the pretraining stage, primarily by filtering unsafe data or rewriting it into safer forms.

By Jinhan Li, Kexian Tang, Yihan Xu, Zhuorui Ye, Kaifeng Lyu

llms fine-tuning safety

arXiv AI

Jun 2

Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning

arXiv:2503. 11832v5 Announce Type: replace Abstract: Recent vision language models (VLMs) have made remarkable strides in generative modeling with multimodal inputs, particularly text and images.

By Yiwei Chen, Yuguang Yao, Yihua Zhang, Bingquan Shen, Gaowen Liu, Sijia Liu

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Aug 14

HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models

arXiv:2608. 12821v1 Announce Type: new Abstract: Large language models (LLMs) remain vulnerable to harmful requests and jailbreak attacks.

By Fangzhou Chen, Shiji Zhao, Mengyang Wang, Qihui Zhu, Ranjie Duan, Maoxun Yuan, Xingxing Wei

llms rag benchmarks safety

arXiv AI

Jul 7

Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models

arXiv:2607. 02914v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse applications, yet ensuring their simultaneous safety, helpfulness, and trustworthiness remains a persistent challenge.

By Jiyang Guan, Yong Xie, Jun Chen, Jiexi Liu, Zipeng Ye, Defeng Li, Jiayu Shen, Jialing Tao, Hui Xue

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 4

Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning

arXiv:2603. 07445v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often require fine-tuning (FT) to perform well on downstream tasks, but FT can induce safety-alignment drift even when the training dataset contains only benign data.

By Guoli Wang, Haonan Shi, Tu Ouyang, An Wang

llms fine-tuning safety

One email a morning, machine-written

Related stories

Matching Ranks Over Probability Yields Truly Deep Safety Alignment

Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection

Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning

HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models

Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models

Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning