arXiv Machine Learning

Improving Improved Kernel PLS

arXiv:2607. 16138v1 Announce Type: new Abstract: Improved Kernel Partial Least Squares (IKPLS) algorithms 1 and 2 are among the fastest PLS calibration algorithms.

Hugging Face Trending Papers
Aug 11

Improving TensorSketch Using Complex Random Variables

\texttt{TensorSketch} by~\cite{pham2013fast,kar2012random} provides efficient sketching algorithms for high-dimensional polynomial kernels $\vec{x}^{\otimes p} \in \R^{d^p}$. \cite{kar2012random} uses dense Johnson-Lindenstrauss (JL)-type projections with computational cost $O(pDd)$, where $D$ denotes the sketch dimension, whereas~\cite{pham2013fast} extends the sparse \texttt{CountSketch}~\citep{count_sketch} algorithm, yielding a faster algorithm for high-dimensional sparse inputs with running time $O\big(p(\nnz{\vec{x}} + D \log D)\big)$.

arXiv Machine Learning
Sep 22

Distributed Linear Solvers and Data Heterogeneity

arXiv:2304.10640v5 Announce Type: replace-cross Abstract: We consider the problem of solving a large-scale system of linear equations in a distributed/federated setting. The taskmaster solves the sys...

By Boris Velasevic, Rohit Parasnis, Christopher G. Brinton, Navid Azizan
arXiv AI
4d ago

ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs

ThinQuant introduces efficient rotation learning for low‑bit weight and activation quantization of large language models by reducing calibration data through a geometric selection of activations and solving a lower‑dimensional optimization problem via an ADMM algorithm. The method achieves comparable or better quantization performance with dramatically fewer calibration points, completing rotation calibration for Llama‑3‑70B in under 12 minutes and for Llama‑3.1‑405B in just over 2 hours on a single GPU. ThinQuant outperforms existing gradient‑free approaches such as DartQuant and gradient‑based SpinQuant in both speed and perplexity metrics on WikiText‑2.

By Mehdi Makni, Ryan Lucas, Rahul Mazumder