Hugging Face Trending Papers
Mitigating Reasoning-Induced Misalignment via Safety-Direction Penalty
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Aug 25
Mitigating Reasoning-Induced Misalignment via Safety-Direction Penalty
arXiv:2608.23497v1 Announce Type: new Abstract: Reasoning-Induced Misalignment, where fine-tuning on reasoning data containing no harmful content, including mathematics, code, and problem-solving wit...
arXiv AI
Oct 21
Rethinking LLM Fine-Tuning via Weight Space Reparameterization: Preserving Safety during Downstream Adaptation
arXiv:2610.26443v2 Announce Type: replace-cross Abstract: Fine-tuning large language models (LLMs) on downstream tasks often weakens their previously aligned safety behavior, creating an inherent tra...
arXiv AI
2d ago
SAVeR$^2$: Reasoning-based Safety Alignment for Large Reasoning Models via Verifiable Rewards
SAVeR$^2$ is a safety alignment method for Large Reasoning Models that decomposes safety objectives into four fine‑grained verifiable reward signals and uses a difficulty‑aware data selection strategy to stabilize training. It introduces a reasoning‑preserving gradient projection mechanism that resolves conflicts between safety and reasoning gradients, eliminating the safety tax. Experiments on models from 1.5B to 671B show that SAVeR$^2$ improves safety performance and reduces over‑refusal rates without harming general reasoning abilities.
Hugging Face Trending Papers
Sep 28
EOPSA: Efficient On-Policy Self-Distilled Safety Alignment
EOPSA (Efficient On-Policy Self-Distilled Safety Alignment) addresses inefficiencies in On-Policy Self-Distillation (OPSD) for safety alignment by focusing training on safety-critical tokens. It introduces Adaptive Rollout Scheduling, which limits generation length based on a Teacher Rescue Rate metric, and Selective Distillation, which filters out safety-neutral tokens to concentrate gradient updates on safety-pivotal transitions. Experiments on models up to 32B parameters show that EOPSA reduces rollout computation by about 50% and backpropagates through only roughly 2% of tokens, outperforming full-token distillation baselines in safety compliance and reasoning retention.
arXiv AI
Oct 27
EOPSA: Efficient On-Policy Self-Distilled Safety Alignment
arXiv:2609.34519v2 Announce Type: replace Abstract: On-Policy Self-Distillation (OPSD) has emerged as a promising paradigm for safety alignment, delivering dense, token-level supervision by distillin...
arXiv AI
Sep 30
actr: aligning thoughts and responses for multilingual safety in reasoning llms
arXiv:2609.37054v1 Announce Type: new Abstract: Ensuring the safety of reasoning large language models (LLMs) across languages is essential for their reliable deployment. However, when exposed to jai...