On analog neuromorphic hardware, intrinsic device noise is normally an accuracy tax. We ask whether it can instead consolidate memories.
The paper demonstrates that adding stochastic noise to a consolidation rule can improve a neural network’s retention of earlier tasks up to an optimal level, after which performance degrades, forming an inverted‑U relationship. Through simulations on related‑task continual‑learning benchmarks, the authors isolate the conditions that produce this optimum, showing it requires coherent restoration toward consolidated weights and is linked to the noise variance. The study further maps the scope of the effect, noting it depends on shared task structure and diminishes with more tasks, while a single‑seed hardware demonstration is referenced elsewhere.
By Gunner Levi Howe
arXiv:2608. 14634v1 Announce Type: new Abstract: Biological intelligence naturally prevents catastrophic forgetting through Complementary Learning Systems (CLS) theory, a macroscopic consolidation process driven at the local level by synaptic metaplasticity: the continuous, history-dependent neuromodulation of individual synapses.
By Isabelle Aguilar, Zayn Andre Zainal, Omid Kavehei
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
By Tieliang Gong, Zhongbo Zhang, Wen Wen, Yong-Jin Liu
Population gradient flow is a common tool for reasoning about how neural networks adapt, including after pretraining. We show that it can mispredict finite-batch stochastic gradient descent (SGD) qualitatively, and we trace the discrepancy to a specific mechanism.
arXiv:2610. 11475v1 Announce Type: new Abstract: Population gradient flow is a common tool for reasoning about how neural networks adapt, including after pretraining.
By Ruoyu Zhao, Mingxuan Zhang, Jianbo Dai, Jiaqi Wu, Chenyu Zhu, Tong Che