arXiv:2607. 25624v1 Announce Type: new Abstract: Positive quadratic networks admit the low-rank representation f_U(x)=x^top UU^top x, where Uinmathbb{R}^{dtimes r} is identifiable only up to right orthogonal multiplication, representing a rank-r PSD matrix Q=UU^top.
arXiv:2608. 10566v1 Announce Type: cross Abstract: How many directions does a neural representation use to encode a concept?
How many directions does a neural representation use to encode a concept? A common answer repeatedly erases probe directions and reports the stopping count or cumulative removed rank.
arXiv:2610.22655v1 Announce Type: cross Abstract: The linear representation hypothesis holds that pretrained vision models encode generative factors as linearly decodable directions in feature space,...
arXiv:2607. 12616v1 Announce Type: new Abstract: Continuous-time generative frameworks construct probability paths between base and target domains by optimizing time-dependent velocity fields.
arXiv:2608. 06809v1 Announce Type: new Abstract: How can an analyst decide whether a nonlinear dimensionality reduction embedding can be trusted?
The paper studies generative flows that sample from an unnormalized target by training a flow to satisfy a balance condition. It shows that the training loss can certify sampler accuracy, can be driven to zero, and quantifies how fast gradient descent converges, with explicit bounds depending on the backward policy and graph structure. The authors provide theoretical guarantees, test them on finite graphs, and certify each theorem using a Lean 4 development.
The paper presents a comprehensive framework for comparing parameter‑efficient fine‑tuning (PEFT) methods by treating each as a smooth map from a small parameter space into weight space, with a pretrained checkpoint as the base point. It identifies three key invariants—image near the base point, fibre dynamics, and base change—that explain expressivity, training dynamics, and initialization behavior. The authors prove numerous results about common PEFT techniques (e.g., LoRA, Householder adaptation) and classify 382 methods from 138 papers using seven coordinate descriptors.
arXiv:2609.37717v1 Announce Type: new Abstract: Decoder-only transformers are trained only through a terminal next-token prediction loss, yet this loss constrains every intermediate hidden state thro...
arXiv:2607. 14018v1 Announce Type: cross Abstract: We investigate how each component of the Transformer feedforward block architecture design determines how much rank survives across depth at initialization.
arXiv:2410. 10137v5 Announce Type: replace Abstract: We develop Riemannian approaches to variational autoencoders (VAEs) for PDE-type ambient data with regularizing geometric latent dynamics, which we refer to as VAE-DLM, or VAEs with dynamical latent manifolds.
We study what spectral summaries and scalar learning curves reveal about centered quadratic networks. We construct analytic positive input densities with exactly Gaussian one- and two-coordinate margi...