arXiv AI

Safer Content or Firmer Refusals? A Hybrid Perturbation Defense for Alignment under Harmful Fine-tuning

arXiv Computation and Language
Sep 4

Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness

The paper investigates how different post‑training methods—supervised fine‑tuning, reasoning‑augmented fine‑tuning, and preference optimization (ORPO)—affect the internal computation of refusal behavior in language models. Experiments on Llama‑3.1‑8B, Gemma‑2‑9B, and Qwen3‑8B show that reasoning‑augmented training consistently creates a distinct refusal computation across models, while the architecture influences the internal structure and steerability of refusal. None of the studied methods simultaneously achieve a distributed refusal mechanism, preserve general capability, and allow easy corrective edits, indicating that current post‑training approaches are not a fully reliable defense for safety-critical applications.

By Hoang Cuong Nguyen, Mark Dras, Usman Naseem
arXiv AI
Oct 28

Aligning Large Language Models for Safety in African Languages: A Case Study of Hausa

The paper introduces HausaSafeLM, a safety preference dataset for Hausa comprising 10,000 prompts (3,000 human‑corrected and 7,000 machine‑translated) across six harm categories. Using Qwen2.5‑7B‑Instruct, the authors compare supervised fine‑tuning (SFT) and SFT followed by Direct Preference Optimization (DPO) on 500 harmful and 200 benign Hausa prompts, finding that SFT improves refusal fluency and precision—especially with larger, noisier data—while DPO sharpens precision at the cost of linguistic quality. No single approach simultaneously maximizes safety, helpfulness, and fluency, and the study shows that refusal‑rate metrics alone are unreliable indicators of safety in low‑resource languages.

By Valdini Douglace Lemofouet, Blessing Ngozi Uzor, Paula Chikaodinaka Anyanwu, Danielle Blanche Kapsa, Saminu Mohammad Aliyu, Seid Muhie Yimam, Shamsuddeen Hassan Muhammad
arXiv Machine Learning
Jun 5

In-Training Defenses against Emergent Misalignment in Language Models

arXiv:2508. 06249v3 Announce Type: replace Abstract: Fine-tuning lets practitioners repurpose aligned large language models (LLMs) for new domains, yet recent work reveals emergent misalignment (EM): Even a small, domain-specific fine-tune can induce harmful behaviors far outside the target domain.

By David Kacz\'er, Magnus J{\o}rgenv{\aa}g, Clemens Vetter, Esha Afzal, Robin Haselhorst, Lucie Flek, Florian Mai