Past, Future, All at Once: Mitigating Stability-Plasticity Dilemma via Post-hoc JANUS Rectification
Read the original on arXiv AI →The paper introduces JANUS, a post‑hoc weight rectification framework that enforces Parameter Space Orthogonality to prevent catastrophic forgetting when fine‑tuning foundation models. By projecting updates into the Jacobian Null Space and employing a Multi‑step Adaptive Rectification mechanism, JANUS dynamically verifies trust regions and adjusts step sizes. Additional techniques such as ghost projection, ghost orientation comparison, and sequence‑level SVD compression provide temporal and spatial efficiency, enabling JANUS to integrate seamlessly with various fine‑tuning methods and effectively mitigate the stability‑plasticity dilemma.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.