The paper extends prior work on volume sampling by providing a Loewner envelope for the centered coefficient covariance in least‑squares regression with a fixed pool of features and responses. It characterizes when this envelope is tight, linking tightness to strict spectral properties of residuals, and introduces a residual‑augmented change of measure to derive a one‑sided slack bound. The results also offer geometric insights at the boundary and demonstrate non‑vacuous certificates through frozen‑feature examples, focusing on conditional centered, full‑Gram‑whitened covariance rather than population generalization.
By Kihun Rhee
The paper investigates volume‑sampled linear readouts with fixed feature pools and responses, focusing on the randomness introduced solely by subset selection. It establishes a globally sharp Loewner envelope for centered, full‑Gram‑whitened coefficient covariance and characterizes when a positive geometric margin exists versus when it vanishes, using conditions on residual covariance slack and a pairwise Naimark‑complement minor test. The results provide explicit geometric boundaries and conservative certificates for strictness and variance terms in fixed‑query squared loss, offering a design‑specific phase characterization for this randomized linear‑readout primitive.
By Kihun Rhee
We study ridge regression from exactly $s$ distinct rows of a fixed design. Responses are fixed, and only the subset is random. The determinant law and selected ridge fit share one positive definite p...
arXiv:2610.07721v1 Announce Type: cross
Abstract: We study ridge regression from exactly $s$ distinct rows of a fixed design. Responses are fixed, and only the subset is random. The determinant law a...
By Kihun Rhee
The paper investigates restricted eigenvalue (RE) bounds for norm‑regularized estimators under heavy‑tailed designs. It shows that the previously conjectured sample‑size law based on Gaussian width fails for heavy‑tailed measurements, due to a phenomenon called simultaneous threshold occupancy. The authors provide explicit counterexamples, derive worst‑case sample‑complexity bounds, and compare the behavior of heavy‑tailed versus Gaussian designs on constant‑width polyhedral descent cones.
By Shi Fu, Huibo Xu, Qixin Zhang, Dacheng Tao
arXiv:2609.09130v1 Announce Type: new
Abstract: An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input...
By Xiaoyu Li, Zhizhou Sha, Jiaojiao Jiang, Junbin Gao, Andi Han
arXiv:2606. 01443v1 Announce Type: cross Abstract: A central difficulty in training Joint-Embedding Predictive Architectures (JEPAs) is preventing representation collapse.
By Triet M. Le
An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input-dependent sparsity in the one-hidden-layer ReLU model of Awasthi et al.
arXiv:2608. 08826v1 Announce Type: new Abstract: Adaptive procedures must work without nuisance information an oracle may use, such as a gradient scale or smoothness index, and robust procedures may have to answer queries whose coordinate and inspection time are chosen only after the data are seen.
By Ibne Farabi Shihab, Adria Binte Habib
arXiv:2607. 21999v1 Announce Type: new Abstract: Long-tailed learning couples two sources of poor generalization: head classes dominate training exposure, while under-represented classes often converge to sharper regions of the loss landscape.
By Jiaxin Deng, Junbiao Pang
The paper introduces a sample-weighted trace-norm geometry for multitask learning, focusing on the end-to-end map from task coefficients to input-space predictors. It derives the exact empirical Rademacher complexity for a fixed-radius class and demonstrates that this measure captures orientation and factorization effects that traditional product-based bounds miss. Experiments on 252 held-out comparisons show that weighted joint nuclear regularization consistently improves population excess error over unweighted nuclear regularization and other baselines.
By Mahdi Mohammadigohari
arXiv:2610.19652v1 Announce Type: new
Abstract: Recent work has sought to transfer first-order nonconvex optimization guarantees to zeroth-order methods through gradient estimators, typically incurri...
By Yuxuan Xia, Xiao-Tong Yuan