The paper proposes a new dimension‑reduction strategy for transfer‑operator models of dynamical systems that focuses on scoring the σ‑algebra generated by coordinates rather than the operator’s spectral span. By using a χ²‑divergence criterion between embedded present and future states, the method guarantees that twice the intrinsic system dimension suffices to capture the full operator spectrum, even for systems with weakly interacting components that would otherwise require exponentially many modes. Experiments on benchmark systems show that this algebraic approach recovers masked components missed by rank‑based methods and enables accurate prediction of those components from few labels.
By Mark Kozdoba, Shie Mannor
arXiv:2608. 11661v1 Announce Type: cross Abstract: A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings.
By Zijian Zhao, Sen Li
arXiv:2609. 12259v1 Announce Type: new Abstract: Matrix-valued memories make rank the natural budget of a learned representation: the number of independent directions a state spans bounds what it can bind, compose, and track.
By Samuel Larson
arXiv:2606. 05131v1 Announce Type: new Abstract: Koopman theory turns nonlinear dynamics into a linear spectral problem.
By Kelan Gray, Finlay Brown, Nicolas Boull\'e, Matthew J. Colbrook
arXiv:2609.01129v1 Announce Type: new
Abstract: We identify a recurrent algebraic regularity in Transformer attention: a sparse subset of effective OV operators $T=OV^\top$ nearly closes under compos...
By Jiming Feng, Junliang Li
arXiv:2608. 15351v1 Announce Type: new Abstract: Nominal LoRA rank is a design parameter; calibrated spectral evidence is a separate inferential quantity.
By Mohammed Ahnouch, Lotfi Elaachak
arXiv:2608. 10566v1 Announce Type: cross Abstract: How many directions does a neural representation use to encode a concept?
By Tingan Jin, Shuhang Dong, Haosong Li, Chung-Hsien Chou
arXiv:2605. 20440v2 Announce Type: replace Abstract: Symmetry is central to the physical sciences, yet machine learning usually captures it only approximately, leaving a residual per-step equivariance error $\varepsilon$ that compounds with depth $M$ as $M\varepsilon$, whereas exact equivariance holds at unbounded depth; we demonstrate this divergence at fourteen orders of magnitude.
By Paulina Hoyos, Shashanka Ubaru, Dongsung Huh, Vasileios Kalantzis, Kenneth L. Clarkson, Misha Kilmer, Haim Avron, Lior Horesh
How many directions does a neural representation use to encode a concept? A common answer repeatedly erases probe directions and reports the stopping count or cumulative removed rank.
Low-rank adaptation (LoRA) is the standard way to fine-tune large models, yet when its two factors are trained independently, the update ignores the geometry of the low-rank weight change it induces. We introduce LoRA-TSD, an optimizer that treats every LoRA step as a tangent vector of the fixed-rank matrix manifold and takes the spectral-norm steepest-descent step of Muon inside that tangent space, mapping the result back to the factors through a retraction native to the LoRA parametrization.
LoRA-TSD introduces a new optimizer for low‑rank adaptation (LoRA) that treats each update as a tangent vector on the fixed‑rank matrix manifold and applies a Muon‑style spectral‑norm steepest‑descent step within that tangent space. The method avoids costly full‑matrix operations and offers a retraction that is up to 2.8× cheaper than previous manifold approaches. The authors prove that their surrogate recovers LoRA‑Pro, identify the Riemannian gradient as the natural stationarity measure, and provide the first global convergence guarantees for both LoRA‑Pro and LoRA‑TSD, achieving superior performance across multiple benchmarks with Llama and Qwen models.
By Dmitrii Andriianov, Andrey Veprikov, Aleksandr Beznosikov
The paper investigates online linear regression with sparse comparators, focusing on feature priming techniques that reweight features using past data. It establishes sparse‑regret lower bounds that invalidate sparse‑logarithmic guarantees for univariate, Pearson, and multivariate priming rules under a past‑only Moore–Penrose protocol, showing ≥Ω(min{T,√d}) clipped regret for unit‑power rules and linear regret for powered rules in high dimensions. The authors also provide tight rank upper bounds for certain priming schemes and present algebraic constructions yielding Ω(min{T,d^{1/4}}) regret for unit‑power multivariate priming, while noting that the exact multivariate frontier remains open.
By Huibo Xu, Shi Fu, Qixin Zhang, Dacheng Tao