arXiv:2610.34015v1 Announce Type: cross
Abstract: Most analyses of neural representations focus on trained endpoints, leaving the process by which internal structure forms comparatively unmeasured. W...
By Sanjukta Krishnagopal
arXiv:2607. 25531v1 Announce Type: cross Abstract: Contemporary machine learning struggles to learn continually, reuse prior knowledge, and expose a comprehensible internal structure.
By Zeki Doruk Erden
arXiv:2606. 17889v1 Announce Type: cross Abstract: Compositional learning systems must balance plasticity, the ability to acquire new knowledge, with stability, the preservation of previously learned components, especially when tasks share structure and risk interference.
By Kathrin Korte, Christian Medeiros Adriano, Joachim Winther Pedersen, Eleni Nisioti, Sebastian Risi
arXiv:2606. 20431v1 Announce Type: new Abstract: Continual learning (CL) systems often forget previously acquired knowledge, yet the mechanisms driving forgetting remain hard to isolate in practice because real datasets entangle many factors.
By Jan Wasilewski, J\k{e}drzej Kozal, Micha{\l} Wo\'zniak, Bartosz Krawczyk
arXiv:2608. 15854v1 Announce Type: new Abstract: Catastrophic forgetting remains a fundamental obstacle to continual learning, where neural networks lose previously acquired knowledge while learning new tasks.
By Maksim A. Kazanskii
The paper introduces a new task called Multimodal Unsupervised Continual Post-Training (MU‑CPT), which allows multimodal large language models (MLLMs) to continuously learn from streaming unlabeled data. It identifies token‑level visual dependence (VD) as essential for MU‑CPT, using its structural distortion to detect cross‑modal forgetting and its heterogeneity to guide new‑task learning. The proposed Visual Dependence‑Aware (VDA) framework includes Visually Constrained Optimal Transport (VC‑OT) to mitigate forgetting and Visually Modulated Adaptation (VMA) to enhance new‑task plasticity, achieving a balance between stability and adaptability in MU‑CPT.
By Kaichen Li, Zhilin Zhu, Jianhao Huang, Zhengqin Lai, Baochen Xiong, Zibo Shao, Yaguang Song, Linhui Xiao, Xiaoshan Yang, Changsheng Xu
arXiv:2609.37836v1 Announce Type: new
Abstract: Neural networks trained toward the same final objective can reach similar predictive performance while retaining internal representations shaped by ear...
By Ertu\u{g}rul Mutlu
The paper investigates how pre‑training on random input‑output mappings (memorization tasks) can transfer to downstream tasks. It discovers two unexpected patterns: equivalent transfer, where each pre‑training epoch saves roughly one fine‑tuning epoch, and non‑equivalent transfer, where pre‑training on a mismatched task can be more efficient than training directly on the downstream task. Ablation studies reveal that transfer consists of a trivial magnitude‑driven effect in the last layer and a non‑trivial structure‑driven effect linked to covariance in other layers.
By Yimiao Yu, Florentin Guth
arXiv:2610.14500v1 Announce Type: cross
Abstract: Many widely used activation functions, including ReLU, GELU, Swish, and Sigmoid, produce predominantly positive outputs, resulting in a nonzero post-...
By Zirui Peng, Heyi Zheng, Ziming Liu
The paper analyzes how temporal correlations in input sequences affect memory formation in linear recurrent neural networks (LRNNs). By solving the learning dynamics for correlated inputs, it shows that correlation introduces a cost to retaining past information, reshaping the learning trajectory and reducing the network’s memory of past inputs. Key findings include a correlation‑dependent threshold for memory retention, the influence of input similarity on memory usefulness, and the emergence of a feedthrough path when zero error is required.
By Arnol Manuel Fokam, Fasseu Sieyondji Akpevwoghene, Edem Fiifi Dawson
FlowGenesis is a framework that lets neural networks self‑organize their architecture during training by adding or removing neurons based on a local statistic called Representation Flux. Neurons with high Flux are split to preserve function, while low‑Flux, low‑activity neurons are pruned. Experiments on image and text classification show that networks starting from different sizes converge to similar neuron counts and performance, and the method also improves continual learning by reducing parameters for new tasks without sacrificing accuracy.
By Maksim A Kazanskii
The paper proposes a drift‑adaptive method for Domain‑Incremental Learning that tracks representation drift for each past domain‑class pair. By measuring how much a concept’s current representation deviates from a fixed anchor, the method assigns stronger preservation to high‑drift pairs and down‑weights highly drifted domains during prototype construction. Experiments on DomainNet, Office‑Home, CORe50, and CDDB‑Hard show that this approach maintains a strong accuracy‑forgetting balance while using less training memory and the smallest storage compared to other methods.
By Hyunmin Kong, Gyutae Oh, Jitae Shin