arXiv Machine Learning

Distributed Linear Solvers and Data Heterogeneity

arXiv Machine Learning
Jul 20

Improving Improved Kernel PLS

arXiv:2607. 16138v1 Announce Type: new Abstract: Improved Kernel Partial Least Squares (IKPLS) algorithms 1 and 2 are among the fastest PLS calibration algorithms.

By Ole-Christian Galbo Engstr{\o}m
arXiv Machine Learning
Sep 17

Revisiting Distributed Sign-Based Variance Reduction

The paper addresses bias introduced by aggregating local signs in distributed sign-based variance reduction methods, which hampers optimal convergence rates. By proposing an unbiased compression of recursive gradient increments to track the global gradient at the server, the authors achieve optimal convergence rates for both nonconvex stochastic and finite-sum optimization. They provide specific rate bounds for α-norms and demonstrate matching sample complexities to centralized settings for finite-sum problems.

By Wei Jiang, Zechao Li, Lijun Zhang
arXiv Machine Learning
Jun 19

Spectral DPPs via NEPv: A Scalable Continuous Relaxation of Determinantal MAP for Diversity-Aware Data Selection

arXiv:2606. 19411v1 Announce Type: new Abstract: Selecting a small, diverse, high-quality subset from a massive pool of candidates is a recurring primitive in modern machine learning -- data curation and coreset selection for training and fine-tuning large models, active-learning batch acquisition, prompt and exemplar selection for in-context learning, retrieval diversification, and experimental design.

By Richard Yi Da Xu
arXiv Machine Learning
4d ago

Averaged Mirror Descent and Dual Gradient Methods: Convergent Algorithms for Entropic Gromov-Wasserstein Problems

The paper studies algorithms for computing the Entropic Gromov-Wasserstein (EGW) distance, a measure of discrepancy between metric measure spaces. It introduces Averaged Mirror Descent (AMD), which averages successive Mirror Descent steps and is proven to converge for any cost function, and shows that a dual gradient method with a fixed step size also converges for arbitrary costs, even when iterations are inexact. Empirical comparisons demonstrate that both AMD and the dual gradient method succeed on cases where classical Mirror Descent fails.

By Joanna Marks, Gabriel Rioux, Riccardo Passeggeri