Refusal geometry reflects refusal training: diverse refusal prefixes can raise stable rank and weaken refusal vector ablation attacks
Read the original on arXiv Machine Learning →The paper investigates how refusal training shapes the internal geometry of language models, showing that activation updates from refusal-completion losses create a distinct low‑dimensional refusal subspace. In a case study on OLMo‑2‑0425‑1B‑Instruct, the authors link the brittleness of refusal directions to repetitive refusal prefixes and demonstrate that using diverse refusal starts can increase the stable rank of gradients, thereby hardening the model against vector‑ablation attacks. The work provides insights into the emergence of safety‑critical features and offers a potential strategy to strengthen refusal robustness.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.