The paper investigates whether the rank of latent matrices in matrix‑chain‑of‑thought (Matrix‑CODI) models influences performance on reasoning tasks. Across multiple training regimes on ProsQA and GSM8K‑Aug, rank‑k projection ablations show flat accuracy curves, indicating that truncating the latent matrix to low rank does not hurt performance. Experiments with various readout architectures—bilinear, bilinear‑plus‑GELU, SVD‑augmented, and quadratic—confirm that rank‑indifference persists even for nonlinear readouts, and a linear probe on the latent matrix underperforms a raw pretrained hidden state.
By Samuel Larson (Pebble ML)
arXiv:2609.17594v1 Announce Type: new
Abstract: Can gradient-based training learn the rank needed to store and compose associations in a matrix memory? In our earlier study, we used a matrix-augmente...
By Samuel Larson
arXiv:2609. 12259v1 Announce Type: new Abstract: Matrix-valued memories make rank the natural budget of a learned representation: the number of independent directions a state spans bounds what it can bind, compose, and track.
By Samuel Larson
arXiv:2609.06341v1 Announce Type: cross
Abstract: Linear algebra provides the framework of concepts (matrix rank, singular value decomposition (SVD), and eigendecomposition) that modern artificial in...
By Anjaneya Teja Sarma Kalvakolanu
Low-rank adaptation fixes the rank of the update, but it does not identify which parts of a trained
write actually carry behavior. We study that question directly and show that behaviorally effectiv...
arXiv:2606. 15036v1 Announce Type: new Abstract: We train a two-layer transformer encoder to classify rational elliptic curves $E/\mathbb{Q}$ of conductor $\leq 10000$ as either rank 0 or rank 1 from the first 128 normalized Frobenius traces.
By Pranav Venkata Konda