arXiv Statistics ML

Dynamics-Based Intrinsic Signal Model for High-Dimensional, Small-Sample Data

arXiv Machine Learning
Sep 30

High-Dimensional Partial Least Squares: Spectral Analysis and Fundamental Limitations

The paper investigates Partial Least Squares (PLS) in high-dimensional settings, focusing on a model where two data matrices share a low-rank latent structure plus individual-specific components. By analyzing the singular vectors of the cross‑covariance matrix with random matrix theory, the authors derive asymptotic characterizations of how well the estimated latent directions align with the true ones. They show that the PLS variant based on Singular Value Decomposition (PLS‑SVD) outperforms separate principal component analysis in detecting the common latent subspace, while also identifying regimes where PLS‑SVD behaves counter‑intuitively or reaches fundamental limits.

By Victor L\'eger, Florent Chatelain
arXiv Statistics ML
Aug 25

Sparse Separable Factor Analysis in the Complex Domain with an Application to Local Field Potential Data

The paper introduces Sparse Separable Factor Analysis (SSFA), a latent factor model designed for complex-valued arrays that preserves amplitude and phase information. SSFA models each mode’s covariance with a low‑rank Hermitian factor structure plus a diagonal residual, applying element‑wise lasso penalties to achieve interpretable, phase‑preserving loadings via complex soft‑thresholding. The method is validated through simulations showing improved covariance estimation over vectorization approaches and applied to local field potential data from mice to compare separability across brain region, frequency, and time, as well as to impute missing recordings due to electrode misplacement.

By Ian Hultman, Kirtikanth Kalapatapu, Yassine Filali, Rainbo Hultman, Sanvesh Srivastava
arXiv Machine Learning
Sep 23

SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA

SuperPCA is a new algorithm for high‑dimensional principal component analysis that exploits an approximate eigenspace of the sample covariance matrix. The authors show that the subspace spanned by several leading eigenvectors contains useful signal information long before individual eigenvectors converge, and they derive posteriori bounds on the angle between this subspace and the true signal subspace. By using only a small number of subsampled coordinates, SuperPCA can achieve up to a ten‑fold improvement in accuracy over classical PCA while reducing data acquisition costs, especially when the signals are approximately sparse.

By Irina-Beatrice Haas, Maike Meier, Yuji Nakatsukasa, Taejun Park
arXiv Machine Learning
Sep 25

Stacked SVD or SVD stacked? A Random Matrix Theory perspective on data integration

The paper compares two popular data‑integration techniques—Stack‑SVD, which concatenates datasets before performing singular value decomposition, and SVD‑Stack, which first decomposes each dataset separately and then aggregates the leading singular vectors. By deriving exact asymptotic performance expressions and phase transitions in a proportional regime, the authors show that neither method uniformly dominates the other when unweighted, but optimally weighted Stack‑SVD outperforms optimally weighted SVD‑Stack when the low‑rank signal is fully shared. They also demonstrate that SVD‑Stack can excel with partially shared components and provide practical algorithms for estimating optimal weights, supported by simulations and genomic experiments.

By Tavor Z. Baharav, Phillip B. Nicol, Rafael A. Irizarry, Rong Ma
arXiv AI
Oct 19

MOSAIC: Module Discovery via Sparse Additive Identifiable Causal Learning for Scientific Time Series

MOSAIC is a sparse temporal variational autoencoder that merges causal representation learning with support recovery to identify interpretable latent modules in scientific time series. It leverages regime‑conditioned temporal variation to discover latent variables and uses an additive decoder to recover a sparse set of observations linked to each latent, providing module‑level interpretability. The method offers theoretical guarantees for ANOVA main‑effect identifiability under smooth mixing functions and demonstrates empirical success across diverse domains such as RNA dynamics, solar wind, ENSO climate, industrial process control, and synthetic tokamak data.

By Shicheng Fan, Nour Elhendawy, Jianle Sun, Ke Fang, Kun Zhang, Yihang Wang, Lu Cheng
arXiv Machine Learning
Jun 9

Disentangled Feature Importance

arXiv:2507. 00260v3 Announce Type: replace-cross Abstract: When predictors are statistically dependent, the appropriate definition of feature importance depends on the operational goal.

By Jin-Hong Du, Kathryn Roeder, Larry Wasserman