arXiv Computation and Language

Structure is in the zoom: probing neural symmetry through dimensionality scaling

The paper investigates how the dimensionality of neural activity—measured via the participation ratio—depends on the size of subsamples in both cortical populations and language model layers. By decomposing the scaling exponent into a sampling floor and a shift along a declared axis, the authors isolate structure that aligns with known symmetries, such as direction selectivity in mouse V1 and grammaticality signals in language models. The method provides an axis‑resolved measurement that can be applied to any array or Hugging Face model layer without training a probe.

arXiv Machine Learning
Nov 6

Reading a neuron's tuning from its correlations with other neurons

The study investigates whether a neuron's tuning properties can be inferred solely from its correlation matrix with other neurons, without any stimulus information. Using a transformer model on the correlation matrix of 512 neurons from mouse visual cortex, the authors successfully decode preferred orientation with a mean error of 25°, and predict receptive-field position with R² values up to 0.48 on a digital twin. The results show that orientation classes are recoverable due to near-circulant relationships in the correlation structure, and that decoding generalizes across animals only when populations are mixed, highlighting the role of inter-animal diversity.

By Francesco L\"assig
arXiv Machine Learning
Sep 21

Certified Topological Interaction in Neural Representations: Exact Tests and the Statistic They Require

The paper introduces the Intersection Euler Characteristic Profile, a topological metric for measuring class overlap in neural representations, and provides exact permutation and sign‑flip tests to assess disentanglement across layers. Using this statistic, the authors analyze 111 networks and 52,650 measurements, finding that disentanglement is depth‑graded, occurs early, and is influenced by training choices such as augmentation and weight decay. The study also demonstrates that the unnormalized mass of the profile predicts test accuracy, while the dimensionless quotient does not outperform simple linear probes.

By Sushovan Majhi
arXiv Computation and Language
Nov 9

Calibrated concept profiles: making the reading of a model's neurons a measurement

The paper introduces calibrated concept profiles, a method that treats direct neuron readings as measurable quantities. By sampling prompts from a defined distribution and generating two independent batteries, the authors compute a per‑neuron profile expressed in the neuron’s own activation spread, complete with noise units and a minimum resolvable change. The approach is applied to OpenCoder‑1.5B across checkpoints spanning two trillion training tokens, providing unit‑scaled, chance‑level‑aware, and checkpoint‑specific construct attribution for each neuron.

By Piotr Wilam
arXiv Computation and Language
Sep 10

Through the Looking Glass: Directly Reading and Writing Transformers

The paper investigates how many transformer components influence a token prediction by measuring the absolute contribution of each unit and channel to the logit. It finds that thousands of components contribute to a single prediction, yet a small subset—often just dozens—carries the majority of the predictive mass. Across models ranging from 124 M to 7 B parameters, the proportion of the model involved in a prediction remains around one to three percent, independent of size, and the study demonstrates that specific components can be directly read and written to modify model behavior without additional training.

By Mark Oskin
arXiv AI
Nov 6

How Finite Brains Generalize Without Bound: Spatial Memory as a Universal Computer

The paper argues that finite brains generalize to arbitrarily long inputs by using spatial memory that grows with input size while the learner’s observations remain finite. It proposes that this memory is structured as cycles—grid modules, place cells, and hippocampal indices—controlled by a prefrontal system that never directly observes the cycles. Experiments show a neural network trained on strings up to 40 symbols achieves high accuracy on longer sequences, outperforming LSTMs and Transformers on several benchmarks and correctly reversing strings up to 15,015 symbols.

By Yuhang He, Si Wu