← Back to all news
arXiv Machine Learning By Tyler Kastner, Nimrod De La Vega, Amir-massoud Farahmand

SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces SingularClip, a method that periodically clips the singular values of all weight matrices to prevent spectral collapse, a newly identified cause of plasticity loss in neural networks trained on nonstationary tasks. The authors empirically and theoretically analyze how growing anisotropy of singular values degrades the ability to fit new targets, and demonstrate that SingularClip outperforms baseline approaches in both continual supervised learning and deep reinforcement learning settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 1

On the Plasticity Collapse in Continual Machine Unlearning

The paper investigates continual machine unlearning, where models must forget data over time. It identifies a fundamental issue called plasticity collapse, where successive unlearning requests cause geometric constraints that saturate parameter space, leading to two failure modes: forward failure (reduced forgetting quality) and backward failure (re‑memorization). Experiments across architectures and datasets confirm that plasticity collapse is a pervasive problem in continual unlearning.

By Yingdan Shi, Xiang Xu, Kaize Ding, Alfred O. Hero, Ren Wang
computer-visionsafety
More like this →