CLEAR is a conditional safety adaptation framework that employs a lightweight hidden‑state gate to continuously control the activation strength of a safety low‑rank adapter. It aims to reduce harmful completions while preserving the performance of the frozen backbone on benign prompts. Experiments on safety and utility benchmarks, including HarmBench and GSM8K, show that CLEAR significantly lowers HarmBench ASR and improves utility compared to globally applied safety tuning methods such as SFT or standard LoRA.
arXiv:2506. 08473v4 Announce Type: replace Abstract: Fine-tuning large language models (LLMs) improves performance but introduces critical safety vulnerabilities: even minimal harmful data can severely compromise safety measures.
arXiv:2508.20766v2 Announce Type: replace-cross Abstract: Safety alignment in Large Language Models (LLMs) often involves mediating internal representations to refuse harmful requests. Recent researc...
arXiv:2606. 02530v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) with human values often degrades their general capabilities, termed the alignment tax.
arXiv:2511. 00382v2 Announce Type: replace-cross Abstract: Organizations increasingly adapt Large Language Models (LLMs) from public repositories such as HuggingFace to downstream tasks.
DNAlign is a lightweight alignment framework that uses control‑theoretic optimization and null‑space projection to steer large language models toward safe behavior while preserving core knowledge and response quality. By treating the LLM as a dynamic system, it introduces controllable perturbations that are restricted to a harmful‑related subspace derived from neutral hidden states, and a value function trained on human preference data adaptively optimizes these control signals. Extensive evaluations across multiple LLM backbones show that DNAlign consistently reduces harmful outputs while maintaining fluency, coherence, and factual utility, outperforming prior alignment baselines without sacrificing generation diversity.
The paper investigates why large language models lose safety alignment after benign fine‑tuning. It argues that safety alignment relies on a low‑rank, output‑routing geometry that becomes flatter during fine‑tuning, and that after only 100 benign examples this routing is sharpened in output‑side MLPs, leading to fragile safety while general performance remains relatively intact. Techniques like LoRA and ASAM can delay this collapse by reducing output‑side sharpness, but their effectiveness diminishes with larger fine‑tuning scales.
arXiv:2606. 06519v1 Announce Type: new Abstract: Open-weight LLMs are increasingly fine-tuned into customized assistants, but downstream fine-tuning can weaken safety alignment and make models more vulnerable to malicious prompts, even when the training data is not intentionally harmful.
arXiv:2606. 09388v1 Announce Type: new Abstract: Deploying safe large language models (LLMs) on resource-constrained edge devices presents a critical challenge: while dual-model systems combining LLMs with guard models provide effective safety guarantees, their substantial memory and computational demands make them prohibitively expensive for on-device deployment.
arXiv:2609.36862v1 Announce Type: cross Abstract: Fine-tuning-as-a-service lets users adapt a safety-aligned language model to their own data, but it also creates a harmful fine-tuning attack surface...
arXiv:2603. 07445v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often require fine-tuning (FT) to perform well on downstream tasks, but FT can induce safety-alignment drift even when the training dataset contains only benign data.
The paper introduces a modular correction framework for large language models that uses Activated LoRA adapters and a context-aware routing mechanism to mitigate harmful outputs. By allowing expert adapters to activate mid-sequence without invalidating the KV cache, the system achieves low-latency, targeted correction during generation. Experiments show improved alignment on safety benchmarks while maintaining task performance, presenting a lightweight, scalable approach to safer LLM deployments.
arXiv:2607. 16242v1 Announce Type: cross Abstract: Fine-Tuning-as-a-Service (FTaaS) platforms let users train large language models (LLMs) on customized tasks, but this pipeline could erode models' safety alignment.