arXiv:2406. 01514v4 Announce Type: replace-cross Abstract: We introduce a training-free safety enhancement method for aligning large language models (LLMs) without the need for supervised fine-tuning or reinforcement learning from human feedback.
arXiv:2601. 07155v3 Announce Type: replace-cross Abstract: Knowledge distillation (KD) is a widely adopted technique for transferring knowledge from large language models to smaller student models; however, conventional supervised KD often suffers from a distribution mismatch between training and inference.
arXiv:2606. 12342v1 Announce Type: cross Abstract: Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language.
arXiv:2606. 02530v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) with human values often degrades their general capabilities, termed the alignment tax.
arXiv:2603. 07445v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often require fine-tuning (FT) to perform well on downstream tasks, but FT can induce safety-alignment drift even when the training dataset contains only benign data.
LLaVAFlow is an information‑theoretic distillation framework designed to preserve cross‑modal alignment in Multimodal Large Language Models during visual instruction tuning. It compresses the mutual information between extracted relations and MLLM embeddings to refine alignment flow, and then maximizes mutual information between pretrained and fine‑tuned alignment flows to transfer compact alignment information. Experiments demonstrate that LLaVAFlow effectively maintains alignment flow, improving downstream performance and generalization.
The paper introduces ACTD, an Anchor-Based Cross-Tokenizer Distillation method that aligns vocabularies and sequences to transfer reasoning capabilities from large language models to smaller students. It uses a novel anchor loss with residual regularization to reduce alignment noise and extends the approach to multiple teachers. Experiments on five reasoning benchmarks with three teachers show state‑of‑the‑art results, with the multi‑teacher variant outperforming existing baselines.
Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both models to share a vocabulary, which rules them out for the cross-family specialists where safety is most degraded.
arXiv:2604. 03873v4 Announce Type: replace Abstract: Black-box knowledge distillation for large language models presents a strict trade-off.
arXiv:2606. 11201v1 Announce Type: cross Abstract: The wide deployment of LLMs has made model alignment necessary to make newly trained models safely and effectively respond to user instructions.
arXiv:2509. 26169v2 Announce Type: replace Abstract: Alignment of large language models remains a central challenge in natural language processing.
arXiv:2607. 22676v1 Announce Type: new Abstract: Post-training is a key mechanism for adapting large language models to downstream tasks.
arXiv:2608. 05168v1 Announce Type: new Abstract: Large language models often fail on reasoning tasks despite possessing the capability to solve them.