arXiv AI
Correct, Don't Delete: Mitigating Emergent Misalignment with Corrective Supervision
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
3d ago
ContextLift: Breaking the Spurious Correlation Behind Conditional Emergent Misalignment
ContextLift is a counterfactual augmentation technique that mitigates conditional emergent misalignment in language models. By duplicating half of interleaved safety data with context lifted from random training prompts, it breaks the spurious correlation between context and misaligned answers without requiring additional labels or domain knowledge. Experiments on Qwen2.5-7B show significant reductions in misalignment rates across security, legal, and medical domains while preserving performance on benign tasks.
arXiv AI
Sep 16
TAME: Token Attribution and Masking for Emergent misalignment
TAME (Token Attribution and Masking for Emergent misalignment) is a three‑stage framework that identifies which training tokens drive harmful behavior in fine‑tuned language models. It first scores tokens by how much fine‑tuning increases their likelihood, then characterizes patterns among high‑attribution tokens, and finally validates them by masking during training. Experiments on Llama and Qwen show that masking the top‑attribution tokens reduces emergent misalignment by 23‑ to 36‑fold, while random masking has no effect.
Hugging Face Trending Papers
Sep 15
TAME: Token Attribution and Masking for Emergent misalignment
Fine-tuning an aligned language model on narrow, flawed data can induce harmful behavior far outside the training domain, known as emergent misalignment (EM). Prior work has localized EM in model weig...
arXiv AI
Sep 25
Sequential knowledge editing breaks a model's ability to tell good evidence from bad, without costing it accuracy
The paper investigates how sequential knowledge editing can degrade a language model’s ability to discern reliable evidence from unreliable evidence without affecting overall accuracy. Using a conservatively tuned LoRA on Qwen2.5‑7B‑Instruct, the authors show that after 1,000 edits the model’s arbitration score for untouched facts drops by 36%, leading to higher error rates on its most confident decisions, while MMLU accuracy remains unchanged. The study also finds that in some model‑method combinations, sequential edits can reduce MMLU to chance levels even though edit success and locality remain perfect.
arXiv AI
Oct 5
Not Until the Evidence Says So: Teaching LLM Investigators When to Close a Case
arXiv:2610.03190v1 Announce Type: cross Abstract: Accident, defect and outage investigations end with a decision that ordinary question answering never faces: whether the evidence gathered so far is...
arXiv AI
3d ago
Human Interventions as Labels: Localizing, Steering, and (Sometimes) Editing the Computations Behind LLM Agent Failures
The paper treats human interventions during LLM agent failures as labels that pinpoint the internal computation that went wrong, rather than merely training data. In a controlled environment, the authors analyze one failure type—over‑asking a clarifying question when the request is already clear—using rule‑based overseer labels validated by blind human judgment. They find that the decision to ask versus act can be causally localized across several models, that over‑asking exhibits distinct mechanistic signatures, and that steering or weight consolidation can mitigate the issue, though outcomes vary by model family.