Preserving Utility in Continual LLM Unlearning via Orthogonal Projections

The paper introduces Continual Orthogonal Projection (COP), a gradient‑projection method that limits each unlearning update to the orthogonal complement of previously cached activation subspaces, thereby preventing interference between successive deletions. COP requires only a low‑rank basis per layer, avoiding the need for past gradients or previous forget sets, and can be applied to any differentiable‑loss unlearning method without altering its objective. Experiments on TOFU and RWKU benchmarks demonstrate that COP maintains model utility across sequential unlearning requests while preserving forgetting effectiveness and preventing collapse of standard objectives.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

• llms
• benchmarks