The paper investigates whether gradient-based training can learn the necessary rank to store and compose associations in a matrix memory. By training on $K$ key‑value bindings that require rank at least $K$ for exact linear recovery, the authors show that the learned effective rank increases with $K$ and that recovery sharply transitions near $k=K$ when rank is capped. Experiments demonstrate high recovery rates (cosine similarity >0.9) across multiple seeds and show that the trained operator approximates the ideal cycle on the entity subspace, though performance declines with larger matrix dimensions when encoder width is fixed.
By Samuel Larson
arXiv:2608.28150v2 Announce Type: replace
Abstract: How much matrix rank is required to preserve every bounded value output of normalized softmax attention? We study the unrestricted maximum-row-\(\e...
By Yuhe Sui, Jianing Zhang, Yingzhi Tang
The paper introduces State‑Conditioned Minimal Sufficient Evidence Recovery (SER), a method that, given a coding agent’s current state, reconstructs a compact set of evidence passages that collectively provide all facts needed for the agent’s next decision. Using the SERBench dataset of 500 held‑out states from 45 repositories, the authors show that their MSS‑Complement approach recovers a complete evidence set for 73.0 % of states with five items and 80.6 % with eight, outperforming baseline ranking methods. The study also demonstrates that this set‑level policy improves downstream performance on AMA‑Bench and highlights the importance of retrieving missing facts rather than merely re‑ranking similar passages.
By Zhexi Feng, Ruiyi Zhang, Yongbo Yang, Pengtao Xie
arXiv:2609.16183v1 Announce Type: new
Abstract: Fixed-state recurrences--linear attention and state-space models--are reported to lag behind attention on associative recall, but whole-architecture co...
By Julian Boesch, Andrew Wee
The paper proposes a new dimension‑reduction strategy for transfer‑operator models of dynamical systems that focuses on scoring the σ‑algebra generated by coordinates rather than the operator’s spectral span. By using a χ²‑divergence criterion between embedded present and future states, the method guarantees that twice the intrinsic system dimension suffices to capture the full operator spectrum, even for systems with weakly interacting components that would otherwise require exponentially many modes. Experiments on benchmark systems show that this algebraic approach recovers masked components missed by rank‑based methods and enables accurate prediction of those components from few labels.
By Mark Kozdoba, Shie Mannor
arXiv:2605. 05189v2 Announce Type: replace-cross Abstract: How many key-value associations can a $d\times d$ linear memory store?
By Nicholas Barnfield, Juno Kim, Eshaan Nichani, Jason D. Lee, Yue M. Lu
The paper investigates whether the rank of latent matrices in matrix‑chain‑of‑thought (Matrix‑CODI) models influences performance on reasoning tasks. Across multiple training regimes on ProsQA and GSM8K‑Aug, rank‑k projection ablations show flat accuracy curves, indicating that truncating the latent matrix to low rank does not hurt performance. Experiments with various readout architectures—bilinear, bilinear‑plus‑GELU, SVD‑augmented, and quadratic—confirm that rank‑indifference persists even for nonlinear readouts, and a linear probe on the latent matrix underperforms a raw pretrained hidden state.
By Samuel Larson (Pebble ML)
arXiv:2607. 16524v1 Announce Type: new Abstract: Cooperative multi-agent RL systems routinely use team-averaged rewards, a feedback-attribution choice that gives each agent the team outcome regardless of its individual contribution.
By Tasha Pais, Richard Higgins
arXiv:2609.32002v2 Announce Type: replace
Abstract: The rank $r$ in LoRA is widely treated as a capacity control: a smaller rank is assumed to yield a simpler model that generalizes better. We show t...
By Babak Barazandeh
arXiv:2606. 07596v1 Announce Type: new Abstract: Fine-tuning often introduces spurious correlations alongside task knowledge, causing systematic failures on underrepresented groups.
By Edward Sun, Dmitrii Troitskii
arXiv:2610.19033v1 Announce Type: new
Abstract: Low-rank adaptation (LoRA) applies every rank-one component of its update to every token, and mixture-of-experts variants replace this with a learned r...
By Mei Okonkwo, Owen Castellanos, Ingrid Solberg, Yusuf Demirel
arXiv:2609. 16391v1 Announce Type: cross Abstract: Weight-only post-training quantization is the cheapest way to shrink a retrieval embedder, and the received advice for applying it -- protect the embedding table, allocate bits by module sensitivity, prefer a ranking-aware objective over weight reconstruction -- was carried into LLM quantization largely intact.
By Hyojung Han