The paper investigates whether gradient-based training can learn the necessary rank to store and compose associations in a matrix memory. By training on $K$ key‑value bindings that require rank at least $K$ for exact linear recovery, the authors show that the learned effective rank increases with $K$ and that recovery sharply transitions near $k=K$ when rank is capped. Experiments demonstrate high recovery rates (cosine similarity >0.9) across multiple seeds and show that the trained operator approximates the ideal cycle on the entity subspace, though performance declines with larger matrix dimensions when encoder width is fixed.
By Samuel Larson
arXiv:2608.28150v2 Announce Type: replace
Abstract: How much matrix rank is required to preserve every bounded value output of normalized softmax attention? We study the unrestricted maximum-row-\(\e...
By Yuhe Sui, Jianing Zhang, Yingzhi Tang
The paper introduces State‑Conditioned Minimal Sufficient Evidence Recovery (SER), a method that, given a coding agent’s current state, reconstructs a compact set of evidence passages that collectively provide all facts needed for the agent’s next decision. Using the SERBench dataset of 500 held‑out states from 45 repositories, the authors show that their MSS‑Complement approach recovers a complete evidence set for 73.0 % of states with five items and 80.6 % with eight, outperforming baseline ranking methods. The study also demonstrates that this set‑level policy improves downstream performance on AMA‑Bench and highlights the importance of retrieving missing facts rather than merely re‑ranking similar passages.
By Zhexi Feng, Ruiyi Zhang, Yongbo Yang, Pengtao Xie
arXiv:2609.16183v1 Announce Type: new
Abstract: Fixed-state recurrences--linear attention and state-space models--are reported to lag behind attention on associative recall, but whole-architecture co...
By Julian Boesch, Andrew Wee
The paper proposes a new dimension‑reduction strategy for transfer‑operator models of dynamical systems that focuses on scoring the σ‑algebra generated by coordinates rather than the operator’s spectral span. By using a χ²‑divergence criterion between embedded present and future states, the method guarantees that twice the intrinsic system dimension suffices to capture the full operator spectrum, even for systems with weakly interacting components that would otherwise require exponentially many modes. Experiments on benchmark systems show that this algebraic approach recovers masked components missed by rank‑based methods and enables accurate prediction of those components from few labels.
By Mark Kozdoba, Shie Mannor
arXiv:2605. 05189v2 Announce Type: replace-cross Abstract: How many key-value associations can a $d\times d$ linear memory store?
By Nicholas Barnfield, Juno Kim, Eshaan Nichani, Jason D. Lee, Yue M. Lu