arXiv Machine Learning

Behavioral Convergence Without Representational Convergence: Persistent Training-History Dependence in Neural Networks

arXiv Machine Learning
Oct 8

When does a network's training history predict its future learning better than its current state? Evidence from a response probe and a forecasting screen

The study investigates whether a neural network’s training history can better predict its future learning than its current state. Using small multilayer perceptrons trained under 42 different history regimes, the authors measured future learning at four checkpoints via a short probe that replicated the network for 100 updates on a new task. Results show that a low‑dimensional history state does not outperform a calibrated model of the current state for short‑term prediction, but in a companion screen of 1,560 synthetic regression runs, history models forecast final error better than current validation error after 12 epochs and remain comparable until 48 epochs.

By Martin Hofmann, Patrick M\"ader
arXiv Machine Learning
Oct 19

Error Correction in Trained TTT Layers

arXiv:2610.16268v2 Announce Type: replace Abstract: Test-time-training (TTT) layers update fast-weight memory while processing an input sequence. A gradient-based update, however, does not by itself...

By Yuhe Hu
arXiv Machine Learning
Oct 7

What Must Replay Preserve? Separating Correctable Bias from Class Correspondence

The paper investigates what information must be preserved in replay buffers for class‑incremental learning. By treating cached predictions as temporally heterogeneous supervision, the authors separate classes known at storage time from those learned later, and evaluate the impact of deleting logit matching. Experiments on CIFAR‑100 with DER++ show that a simple task‑level offset can largely correct the cost of removing later‑class matching, while the cost of disrupting class correspondence remains.

By BoRen Deng, Xiangyue Ma, Chenglong Li, Xiaoting Du