arXiv AI By Yuhang He, Si Wu

How Finite Brains Generalize Without Bound: Spatial Memory as a Universal Computer

Read the original on arXiv AI →

The paper argues that finite brains generalize to arbitrarily long inputs by using spatial memory that grows with input size while the learner’s observations remain finite. It proposes that this memory is structured as cycles—grid modules, place cells, and hippocampal indices—controlled by a prefrontal system that never directly observes the cycles. Experiments show a neural network trained on strings up to 40 symbols achieves high accuracy on longer sequences, outperforming LSTMs and Transformers on several benchmarks and correctly reversing strings up to 15,015 symbols.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Sep 2

How Temporal Correlations Shape Memory in Linear Recurrent Neural Networks

The paper analyzes how temporal correlations in input sequences affect memory formation in linear recurrent neural networks (LRNNs). By solving the learning dynamics for correlated inputs, it shows that correlation introduces a cost to retaining past information, reshaping the learning trajectory and reducing the network’s memory of past inputs. Key findings include a correlation‑dependent threshold for memory retention, the influence of input similarity on memory usefulness, and the emergence of a feedthrough path when zero error is required.

By Arnol Manuel Fokam, Fasseu Sieyondji Akpevwoghene, Edem Fiifi Dawson
arXiv Computation and Language
Nov 9

Representational drift in a code model: behaviour settles, which neurons carry it does not

The study investigates representational drift in the OpenCoder‑1.5B language model by tracking how neurons encode 169 Python constructs during pretraining on two trillion tokens. While the loss for 16 grammar‑fixing constructs quickly reaches a floor, the specific neurons that carry these constructs continue to change throughout training, with no stable carrier set emerging even at the final checkpoint. The authors find that removing the identified carriers incurs a measurable performance penalty, yet the carrier sets remain dynamic, challenging the hypothesis that they settle over time.

By Piotr Wilam