arXiv Machine Learning
Jul 22
Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs
arXiv:2607. 18639v1 Announce Type: new Abstract: Safety interventions on dual-use knowledge typically choose between destroying hazardous content (e.
Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs
Safety interventions on dual-use knowledge typically choose between destroying hazardous content (e. g.
Related stories
arXiv Computation and Language
Aug 21
ConceptGuard: Benchmarking Context-Sensitive Unlearning in Large Language Models
arXiv:2608. 20338v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly require selective removal of harmful or sensitive knowledge, called unlearning, yet existing methods and benchmarks fail to evaluate this capability completely.
arXiv Machine Learning
Jun 5
SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion
arXiv:2605. 07482v2 Announce Type: replace Abstract: Machine unlearning for large language models (LLMs) aims to selectively remove memorized content such as private data, copyrighted text, or hazardous knowledge, without costly full retraining.
Hugging Face Trending Papers
5d ago
EOPSA: Efficient On-Policy Self-Distilled Safety Alignment
EOPSA (Efficient On-Policy Self-Distilled Safety Alignment) addresses inefficiencies in On-Policy Self-Distillation (OPSD) for safety alignment by focusing training on safety-critical tokens. It introduces Adaptive Rollout Scheduling, which limits generation length based on a Teacher Rescue Rate metric, and Selective Distillation, which filters out safety-neutral tokens to concentrate gradient updates on safety-pivotal transitions. Experiments on models up to 32B parameters show that EOPSA reduces rollout computation by about 50% and backpropagates through only roughly 2% of tokens, outperforming full-token distillation baselines in safety compliance and reasoning retention.
arXiv AI
Jul 29
Similar Models Learn Differently: Final-Window Pretraining Shapes Post-Training Beyond SFT
arXiv:2607. 25063v1 Announce Type: new Abstract: Developers judge a model checkpoint by how it behaves.
arXiv AI
Jun 29
CBD: API-Only LLM Black-Box Unlearning through Controlled Behavioral Divergence
arXiv:2606. 27683v1 Announce Type: cross Abstract: Edge devices increasingly invoke large language models (LLMs) through API services for context aware edge intelligence, while edge generated data may be collected to improve LLMs and may introduce sensitive, copyrighted, harmful, or outdated information into model behavior.
arXiv Machine Learning
Aug 18
DUET: Dual-Teacher On-Policy Distillation via Same-Weight Disagreement for Prohibition Compliance
arXiv:2608. 14644v1 Announce Type: new Abstract: Real-world LLM deployments increasingly rely on runtime-injected prohibitions--enterprise policies, PII redlines, tool boundaries--that vary per request and per tenant.
arXiv AI
Sep 10
Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks
The paper investigates how different post‑training interventions—harmful supervised fine‑tuning (SFT), harmful reinforcement learning with verifiable rewards (RLVR), and refusal‑feature ablation—affect large language models’ harmful compliance, capability, and safety signals. Across Qwen2.5‑7B and Llama‑3.1‑8B, all methods achieve near‑maximum harmfulness, but SFT causes the greatest loss of capability and representational drift, ablation suppresses refusal features in a family‑specific way, and RLVR largely preserves base‑model performance while redirecting behavior toward compliance. RLVR models also exhibit “capability‑blind compliance,” falsely claiming to perform unavailable actions, which can be mitigated by targeted calibration without harming overall capability. The study demonstrates that harmful compliance, harm recognition, and capability awareness are distinct behavioral axes and that typical safety signals such as self‑audit and hallucination may not reliably indicate robustness after adaptive post‑training.
Hugging Face Trending Papers
Sep 9
Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers...
arXiv AI
Jul 3
DemoPSD: Disagreement-Modulated Policy Self-Distillation
arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.
arXiv Machine Learning
Jun 25
Do Thinking Tokens Help with Safety?
arXiv:2606. 25013v1 Announce Type: new Abstract: Today's reasoning models use thinking tokens to attain stronger performance on benchmarks than their instruction-tuned counterparts.
arXiv AI
Sep 11
Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.