arXiv Machine Learning
Jun 10
Bidirectional Random Projections
arXiv:2606. 10377v1 Announce Type: cross Abstract: This paper analyzes bidirectional random projections for ordinary least squares (OLS) regression under the fixed design setting.
Distributed Sketching on Data Partitions for OLS Regression
arXiv:2607. 07888v1 Announce Type: new Abstract: This paper studies distributed sketching for ordinary least squares (OLS) regression, an approach that distributes small sketches of a large data set over multiple machines to separately construct OLS estimators and average them.
Related stories
arXiv Machine Learning
Jul 7
It's all In the (Exponential) Family: An Equivalence between Maximum Likelihood Estimation and Control Variates for Sketching Algorithms
arXiv:2601. 22378v3 Announce Type: replace-cross Abstract: Maximum likelihood estimators (MLE) and control variate estimators (CVE) have been used in conjunction with known information across sketching algorithms and applications in machine learning.
Hugging Face Trending Papers
Jul 27
The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
Over-parameterized linear regression has been widely studied over the last decade. However, most existing works assume that the covariates are independent and that their covariance matrices are non-degenerate.
arXiv Machine Learning
Jun 10
A Sketch-and-Project Analysis of Subsampled Natural Gradient Algorithms
arXiv:2508. 21022v3 Announce Type: replace Abstract: Subsampled natural gradient descent (SNG) has been used to enable high-precision scientific machine learning, but standard analyses based on stochastic preconditioning fail to provide insight into realistic small-sample settings.
arXiv Machine Learning
Jul 28
The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
arXiv:2607. 24041v1 Announce Type: cross Abstract: Over-parameterized linear regression has been widely studied over the last decade.
arXiv Machine Learning
Jul 13
Near-optimal Delta-convex Estimation of Lipschitz Functions
arXiv:2511. 15615v2 Announce Type: replace-cross Abstract: This paper presents a tractable algorithm for estimating an unknown Lipschitz function from noisy observations and establishes an upper bound on its convergence rate.
arXiv Machine Learning
Jul 2
Distributionally Robust Linear Regression With Block Lewis Weights
arXiv:2607. 00252v1 Announce Type: new Abstract: We present an algorithm for the group distributionally robust (GDR) least squares problem.
arXiv Machine Learning
Jun 16
SILAGE: Memory-Efficient, Full-Gradient-Free Nonconvex Optimization for Nested Finite Sums
arXiv:2606. 15832v1 Announce Type: new Abstract: Empirical risk minimization on massive datasets naturally exhibits a nested double finite-sum structure, where $N=nm$ total samples are logically or physically partitioned into $n$ blocks of size $m$ (e.
arXiv Machine Learning
Jun 2
Variable Clustering via Distributionally Robust Nodewise Regression
arXiv:2212. 07944v4 Announce Type: replace Abstract: We study a multi-factor block model for variable clustering and connect it to regularized subspace clustering through a distributionally robust version of nodewise regression.
Hugging Face Trending Papers
Aug 11
Improving TensorSketch Using Complex Random Variables
\texttt{TensorSketch} by~\cite{pham2013fast,kar2012random} provides efficient sketching algorithms for high-dimensional polynomial kernels $\vec{x}^{\otimes p} \in \R^{d^p}$. \cite{kar2012random} uses dense Johnson-Lindenstrauss (JL)-type projections with computational cost $O(pDd)$, where $D$ denotes the sketch dimension, whereas~\cite{pham2013fast} extends the sparse \texttt{CountSketch}~\citep{count_sketch} algorithm, yielding a faster algorithm for high-dimensional sparse inputs with running time $O\big(p(\nnz{\vec{x}} + D \log D)\big)$.
arXiv Machine Learning
Jun 16
Active Learning with Low-Rank Structure for Data Selection
arXiv:2606. 16045v1 Announce Type: new Abstract: In the data selection problem, the objective is to choose a small, representative subset of data that can be used to efficiently train a machine learning model.
arXiv Machine Learning
Jun 30
Nonlinear mixture model motivated subspace clustering
arXiv:2606. 29261v1 Announce Type: new Abstract: We derive the linear union-of-subspaces (UoS) model for subspace clustering (SC) from the nonlinear mixture model (NMM) used in blind source separation (BSS) to represent a D-dimensional observation vector as an unknown multivariate nonlinear mapping of C latent variables.