arXiv Statistics ML By Samuel Erickson, Tobias Ryd\'en

Non-asymptotic Estimation of Sparse Partial Correlation via Joint Partial Regression

Read the original on arXiv Statistics ML →

The paper introduces a two‑stage method for estimating sparse high‑dimensional partial correlation matrices and inverse covariance matrices with non‑asymptotic guarantees. It leverages the link between inverse covariance and linear regression while ensuring positive semi‑definiteness, and provides an efficient proximal splitting algorithm for computation. The authors demonstrate the method’s effectiveness on synthetic and real‑world datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Statistics ML.

arXiv Statistics ML
5d ago

Unifying Sparse Estimation of Precision and Covariance Matrices

The paper proposes a unified framework for estimating sparse precision and covariance matrices by reparametrizing the last column of either matrix. This reparametrization reduces the likelihood optimization to a one‑dimensional minimization followed by a penalized multiple regression, applicable to any penalty separable in off‑diagonal entries and linear on the diagonal. When the penalty is the α1 norm, the method simultaneously recovers the graphical Lasso and the covariance graphical Lasso algorithms.

By Aramayis Dallakyan, Mohsen Pourahmadi
arXiv Machine Learning
Sep 30

High-Dimensional Partial Least Squares: Spectral Analysis and Fundamental Limitations

The paper investigates Partial Least Squares (PLS) in high-dimensional settings, focusing on a model where two data matrices share a low-rank latent structure plus individual-specific components. By analyzing the singular vectors of the cross‑covariance matrix with random matrix theory, the authors derive asymptotic characterizations of how well the estimated latent directions align with the true ones. They show that the PLS variant based on Singular Value Decomposition (PLS‑SVD) outperforms separate principal component analysis in detecting the common latent subspace, while also identifying regimes where PLS‑SVD behaves counter‑intuitively or reaches fundamental limits.

By Victor L\'eger, Florent Chatelain
arXiv AI
Sep 28

Adaptive multi-resolution Gaussian processes: Scalable exact inference with naturally data-sparse covariance matrices

The paper introduces an adaptive multi‑resolution Gaussian process framework that achieves scalable, exact inference by constructing a naturally data‑sparse covariance matrix using basis functions anchored directly to samples. By shrinking the support domains of these basis functions, the resulting matrix has limited block sizes, ensuring sparsity and enabling efficient computation of its inverse via a sparse Cholesky algorithm. The authors demonstrate that this approach yields exact inference with training cost ≠≠ O(n log^2 n) and prediction cost ≠≠ O(log^d n), while also improving predictive uncertainties through an augmented basis function.

By Yanchuang Cao, Jun Liu, Tengchao Yu, Heng Yong
arXiv Machine Learning
Sep 25

Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integration

The paper compares two popular data‑integration techniques—Stack‑SVD, which concatenates datasets before performing singular value decomposition, and SVD‑Stack, which first decomposes each dataset separately and then aggregates the leading singular vectors. By deriving exact asymptotic performance expressions and phase transitions in a proportional regime, the authors show that neither method uniformly dominates the other when unweighted, but optimally weighted Stack‑SVD outperforms optimally weighted SVD‑Stack when the low‑rank signal is fully shared. They also demonstrate that SVD‑Stack can excel with partially shared components and provide practical algorithms for estimating optimal weights, supported by simulations and genomic experiments.

By Tavor Z. Baharav, Phillip B. Nicol, Rafael A. Irizarry, Rong Ma