arXiv:2609. 11712v1 Announce Type: cross Abstract: In this paper, we investigate the generalization performance of distributed gradient descent algorithms in a reproducing kernel Hilbert space under a robust loss function $l_{\sigma}$.
arXiv:2402.04691v5 Announce Type: replace-cross Abstract: This study investigates the use of stochastic gradient descent (SGD) to learn operators between general Hilbert spaces. We study weak and str...
arXiv:2405. 11667v2 Announce Type: replace Abstract: Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD.
arXiv:2502.07977v3 Announce Type: replace Abstract: Empirical risk minimization (ERM) is a cornerstone of modern machine learning. This paper focuses on the man-in-the-middle (MITM) attack, wherein a...
The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.
arXiv:2606. 06772v2 Announce Type: replace-cross Abstract: Characterizing the optimization dynamics and statistical performance of over-parameterized deep neural networks (DNNs) remains a central challenge in understanding the remarkable success of deep learning.
The paper investigates the use of Johnson‑Lindenstrauss (JL) transforms in Byzantine‑robust distributed optimization. By leveraging JL’s ability to preserve inner products and distances, worker messages are weighted based on their alignment with a trusted reference gradient, reducing communication costs while maintaining robustness. Experimental results support the theoretical claims.
The growing scale of data and models in machine learning calls for efficient optimization methods. Distributed optimization addresses this challenge, with compression mechanisms helping to reduce comm...
arXiv:1812.09632v3 Announce Type: replace Abstract: We propose a data-driven approach to quantify the uncertainty of models constructed by kernel methods. Our approach minimizes the needed distributi...
arXiv:2504.18184v5 Announce Type: replace Abstract: We consider a class of statistical inverse problems involving the estimation of a regression operator from a Polish space to a separable Hilbert sp...
The paper introduces a distributionally robust method for learning hyperparameters of first‑order convex optimization algorithms. By minimizing a Wasserstein‑robust performance estimation problem over a dataset of problem instances, the approach interpolates between classical learning‑to‑optimize (L2O) and worst‑case PEP design. The authors solve the resulting problem with stochastic gradient descent, provide high‑probability risk bounds, and demonstrate that the learned algorithms outperform both worst‑case optimal and vanilla L2O baselines on logistic regression, LASSO, and linear programming tasks.
arXiv:2608. 06563v1 Announce Type: new Abstract: Machine learning and optimization have advanced together, with practical demands motivating new theory and theoretical breakthroughs enabling new applications.
arXiv:2603. 16481v3 Announce Type: replace Abstract: Non-conservative uncertainty bounds are essential for making reliable predictions about latent functions from noisy data, and thus, a key enabler for safe learning-based control.