arXiv Machine Learning June 4, 2026 By Guoli Wang, Haonan Shi, Tu Ouyang, An Wang

Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning

Read the original on arXiv Machine Learning →

arXiv:2603. 07445v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often require fine-tuning (FT) to perform well on downstream tasks, but FT can induce safety-alignment drift even when the training dataset contains only benign data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 24

RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs

arXiv:2605.01913v2 Announce Type: replace-cross Abstract: Fine-tuning safety-aligned language models for downstream tasks often leads to substantial degradation of refusal behavior, making models vul...

By Sadia Asif, Mohammad Mohammadi Amiri

llms fine-tuning benchmarks safety

arXiv AI

Jun 2

SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment

arXiv:2606. 02530v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) with human values often degrades their general capabilities, termed the alignment tax.

By Hao Li, Jingkun An, Zijun Song, Pengyu Zhu, Rui Li, Hao Wang, Wendi Feng, Yesheng Liu, Lijun Li, Jin-Ge Yao, Lei Sha

llms efficiency benchmarks safety

arXiv AI

Jun 18

Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection

arXiv:2606. 19168v1 Announce Type: new Abstract: To achieve deeper safety alignment for large language models (LLMs), recent efforts have studied how to push safety interventions earlier into the pretraining stage, primarily by filtering unsafe data or rewriting it into safer forms.

By Jinhan Li, Kexian Tang, Yihan Xu, Zhuorui Ye, Kaifeng Lyu

llms fine-tuning safety

arXiv Machine Learning

Jun 11

AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basin

arXiv:2506. 08473v4 Announce Type: replace Abstract: Fine-tuning large language models (LLMs) improves performance but introduces critical safety vulnerabilities: even minimal harmful data can severely compromise safety measures.

By Shuo Yang, Qihui Zhang, Yuyang Liu, Xiaojun Jia, Kunpeng Ning, Jiayu Yao, Jigang Wang, Hailiang Dai, Yibing Song, Li Yuan

llms fine-tuning safety

arXiv AI

Jul 21

TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment

arXiv:2607. 16242v1 Announce Type: cross Abstract: Fine-Tuning-as-a-Service (FTaaS) platforms let users train large language models (LLMs) on customized tasks, but this pipeline could erode models' safety alignment.

By Changyue Li, Jiaming He, Youliang Yuan, Jialin Wu, Boxi Yu, Zhicong Huang, Pinjia He

llms fine-tuning benchmarks safety

arXiv AI

Jul 23

Matching Ranks Over Probability Yields Truly Deep Safety Alignment

arXiv:2512. 05518v2 Announce Type: replace-cross Abstract: Open-source Large Language Models (LLMs) play a critical role in the democratization of AI, yet their "open" nature introduces more avenues for malicious actors to misuse them for harmful purposes.

By Jason Vega, Gagandeep Singh

llms fine-tuning safety

One email a morning, machine-written

Related stories

RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs

SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment

Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection

AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basin

TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment

Matching Ranks Over Probability Yields Truly Deep Safety Alignment