arXiv Computation and Language By Adil Amin

Structure is in the zoom: probing neural symmetry through dimensionality scaling

Read the original on arXiv Computation and Language →

The paper investigates how the dimensionality of neural activity—measured via the participation ratio—depends on the size of subsamples in both cortical populations and language model layers. By decomposing the scaling exponent into a sampling floor and a shift along a declared axis, the authors isolate structure that aligns with known symmetries, such as direction selectivity in mouse V1 and grammaticality signals in language models. The method provides an axis‑resolved measurement that can be applied to any array or Hugging Face model layer without training a probe.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning
Nov 6

Reading a neuron's tuning from its correlations with other neurons

The study investigates whether a neuron's tuning properties can be inferred solely from its correlation matrix with other neurons, without any stimulus information. Using a transformer model on the correlation matrix of 512 neurons from mouse visual cortex, the authors successfully decode preferred orientation with a mean error of 25°, and predict receptive-field position with R² values up to 0.48 on a digital twin. The results show that orientation classes are recoverable due to near-circulant relationships in the correlation structure, and that decoding generalizes across animals only when populations are mixed, highlighting the role of inter-animal diversity.

By Francesco L\"assig
arXiv Machine Learning
Sep 21

Certified Topological Interaction in Neural Representations: Exact Tests and the Statistic They Require

The paper introduces the Intersection Euler Characteristic Profile, a topological metric for measuring class overlap in neural representations, and provides exact permutation and sign‑flip tests to assess disentanglement across layers. Using this statistic, the authors analyze 111 networks and 52,650 measurements, finding that disentanglement is depth‑graded, occurs early, and is influenced by training choices such as augmentation and weight decay. The study also demonstrates that the unnormalized mass of the profile predicts test accuracy, while the dimensionless quotient does not outperform simple linear probes.

By Sushovan Majhi
arXiv Computation and Language
Nov 9

Calibrated concept profiles: making the reading of a model's neurons a measurement

The paper introduces calibrated concept profiles, a method that treats direct neuron readings as measurable quantities. By sampling prompts from a defined distribution and generating two independent batteries, the authors compute a per‑neuron profile expressed in the neuron’s own activation spread, complete with noise units and a minimum resolvable change. The approach is applied to OpenCoder‑1.5B across checkpoints spanning two trillion training tokens, providing unit‑scaled, chance‑level‑aware, and checkpoint‑specific construct attribution for each neuron.

By Piotr Wilam