The study investigates whether a neural network’s training history can better predict its future learning than its current state. Using small multilayer perceptrons trained under 42 different history regimes, the authors measured future learning at four checkpoints via a short probe that replicated the network for 100 updates on a new task. Results show that a low‑dimensional history state does not outperform a calibrated model of the current state for short‑term prediction, but in a companion screen of 1,560 synthetic regression runs, history models forecast final error better than current validation error after 12 epochs and remain comparable until 48 epochs.
By Martin Hofmann, Patrick M\"ader
arXiv:2608.20965v1 Announce Type: new
Abstract: We define an atomic generation fact f=(u,tau,omega,z;rho), recording the origin, realized transformation, concrete occurrence, generated result and rel...
By Mian Wang
arXiv:2609.36081v1 Announce Type: new
Abstract: Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure th...
By Yuantao Deng, Jinnuo Liu, Kaizhen Tan, Yuchen Liu
arXiv:2610.16268v2 Announce Type: replace
Abstract: Test-time-training (TTT) layers update fast-weight memory while processing an input sequence. A gradient-based update, however, does not by itself...
By Yuhe Hu
arXiv:2507. 01414v2 Announce Type: replace Abstract: We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall.
By Sultan Daniels, Dylan Davis, Dhruv Gautam, Wentinn Liao, Gireeja Ranade, Anant Sahai
arXiv:2608. 11690v1 Announce Type: new Abstract: Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.
By Tieliang Gong, Zhongbo Zhang, Wen Wen, Yong-Jin Liu
arXiv:2609.36375v1 Announce Type: new
Abstract: Continual learning is usually studied through mechanisms that preserve old knowledge. We develop Successional Learning Theory (SLT), a mesoscopic accou...
By Shoaib Ahmed Dipu, Md Salman Shamil, Sayeed Shafayet Chowdhury
arXiv:2610.22381v1 Announce Type: new
Abstract: When a fact becomes harder to elicit from a language model, has the model lost it, or has it merely become less accessible? Parameter-updating test-tim...
By Zhe Zhou
arXiv:2605. 30556v2 Announce Type: replace Abstract: CORRECTION (August 2026): the central finding of this paper is not supported.
By Nils Leutenegger
arXiv:2609.38768v1 Announce Type: cross
Abstract: Prior work has shown that neural networks exhibit implicit biases toward low-complexity structure (e.g., spectral bias), memorization dynamics, and c...
By Fujie Gao, Zuyue Zhang, Gang Sun
arXiv:2601. 18699v2 Announce Type: replace Abstract: Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities.
By Gustav Olaf Yunus Laitinen-Fredriksson Lundstrom-Imanov
The paper investigates what information must be preserved in replay buffers for class‑incremental learning. By treating cached predictions as temporally heterogeneous supervision, the authors separate classes known at storage time from those learned later, and evaluate the impact of deleting logit matching. Experiments on CIFAR‑100 with DER++ show that a simple task‑level offset can largely correct the cost of removing later‑class matching, while the cost of disrupting class correspondence remains.
By BoRen Deng, Xiangyue Ma, Chenglong Li, Xiaoting Du