arXiv:2609.27988v1 Announce Type: cross Abstract: Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direc...
The paper investigates whether the rank of latent matrices in matrix‑chain‑of‑thought (Matrix‑CODI) models influences performance on reasoning tasks. Across multiple training regimes on ProsQA and GSM8K‑Aug, rank‑k projection ablations show flat accuracy curves, indicating that truncating the latent matrix to low rank does not hurt performance. Experiments with various readout architectures—bilinear, bilinear‑plus‑GELU, SVD‑augmented, and quadratic—confirm that rank‑indifference persists even for nonlinear readouts, and a linear probe on the latent matrix underperforms a raw pretrained hidden state.
Methods operating on Vision Transformer (ViT) feature spaces typically rely on Euclidean distance or cosine similarity. This assumes that every direction is equally meaningful, but there is no reason...
arXiv:2608. 15351v1 Announce Type: new Abstract: Nominal LoRA rank is a design parameter; calibrated spectral evidence is a separate inferential quantity.
The paper studies Online Kernel Supervised Principal Component Analysis (OKSPCA), which uses random features and an Adam-style orthonormal basis update to optimize a supervised spectral objective. It shows that accurate optimization of this objective does not guarantee accurate population subspace recovery or improved predictive performance, and it provides theoretical results on consistency, concentration, and perturbation of the estimator. Empirical experiments on six benchmarks reveal that replacing the tracker with the exact empirical target does not significantly change regression deficits, while classification-rank models capture most of the terminal objective energy but can exhibit substantial geometric deviation; sample-size studies further separate empirical accuracy from population recovery. The diagnostics also compare computational trade-offs, indicating that exact on-request computation can be faster in classification settings, whereas Adam saves time relative to full thin‑SVD in some dense regression requests, despite persistent geometric error.
The paper investigates whether the rank of matrix-valued latent representations in continuous chain‑of‑thought models influences task accuracy. Experiments on ProsQA and GSM8K‑Aug show that truncating the latent matrix to low rank has negligible effect (within 0.6 pp), and this flatness persists across various readout designs and even in a vanilla GPT‑2 baseline. The results suggest that rank is not a useful structural signal for these models’ reasoning paths.
arXiv:2609.15825v1 Announce Type: new Abstract: A self-supervised encoder is trained once, frozen, and reused through lightweight probes on tasks nobody named at training time; the practitioner's que...
arXiv:2606. 07596v1 Announce Type: new Abstract: Fine-tuning often introduces spurious correlations alongside task knowledge, causing systematic failures on underrepresented groups.
arXiv:2606. 01443v1 Announce Type: cross Abstract: A central difficulty in training Joint-Embedding Predictive Architectures (JEPAs) is preventing representation collapse.
arXiv:2609. 12259v1 Announce Type: new Abstract: Matrix-valued memories make rank the natural budget of a learned representation: the number of independent directions a state spans bounds what it can bind, compose, and track.
COMPLEX is a closed‑form, training‑free embedding for multiparameter persistence modules that provides both an upper and a lower Lipschitz bound, enabling faithful feature representations. By slicing modules along a near‑diagonal net and embedding each slice with the certified PLACE/PALACE landmark map, the method guarantees that separated modules remain separated in the embedding. On Orbit benchmarks and molecular graph tasks, COMPLEX achieves state‑of‑the‑art accuracy, outperforming existing landmark, transformer, and graph‑based approaches.
Parameter-efficient fine-tuning is usually framed as a question of how many parameters to update. Under a severe trainable-state budget, however, where those coefficients act is equally consequential....