arXiv:2607. 10430v1 Announce Type: cross Abstract: Dimensionality reduction has proven powerful for identifying neural manifolds, which are low-dimensional structures underlying high-dimensional neural activity.
By Hardik Rajpal, Dan Goodman
FlowGenesis is a framework that lets neural networks self‑organize their architecture during training by adding or removing neurons based on a local statistic called Representation Flux. Neurons with high Flux are split to preserve function, while low‑Flux, low‑activity neurons are pruned. Experiments on image and text classification show that networks starting from different sizes converge to similar neuron counts and performance, and the method also improves continual learning by reducing parameters for new tasks without sacrificing accuracy.
By Maksim A Kazanskii
Training in artificial neural networks can be viewed as a trajectory evolving through a high-dimensional loss landscape. However, the large number of trainable parameters makes the direct analysis of these dynamics challenging.
arXiv:2603. 01568v2 Announce Type: replace Abstract: Efficient coding theory predicts that biological perceptual systems compress sensory input optimally under resource constraints, with the systematic structure of errors reflecting the geometry of that compression.
By Leyla Roksan Caglar, Pedro A. M. Mediano, Baihan Lin
arXiv:2606. 30384v1 Announce Type: new Abstract: Training in artificial neural networks can be viewed as a trajectory evolving through a high-dimensional loss landscape.
By Pedro Jim\'enez-Gonz\'alez, Miguel C. Soriano, Lucas Lacasa
The paper investigates how information-theoretic and geometric compression unfolds over time in recurrent neural networks, specifically discretized LSTMs. It finds that compression is strongest at the final timesteps of many datasets and attributes this to three mechanisms: class separability, temporal stationarity of shared dynamics, and limited representational capacity. Through synthetic and real-world experiments, the authors demonstrate that these factors jointly determine the extent of temporal compression and its role as a regularizer when excess capacity exists, but can harm generalization when it consumes task-relevant memory.
By Nikolaus Kopp, Franz Pernkopf, Bernhard Geiger