Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
Read the original on arXiv Computation and Language →The paper investigates how different post‑training methods—supervised fine‑tuning, reasoning‑augmented fine‑tuning, and preference optimization (ORPO)—affect the internal computation of refusal behavior in language models. Experiments on Llama‑3.1‑8B, Gemma‑2‑9B, and Qwen3‑8B show that reasoning‑augmented training consistently creates a distinct refusal computation across models, while the architecture influences the internal structure and steerability of refusal. None of the studied methods simultaneously achieve a distributed refusal mechanism, preserve general capability, and allow easy corrective edits, indicating that current post‑training approaches are not a fully reliable defense for safety-critical applications.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.