← Back to all news
arXiv Machine Learning By Alexander Tyurin

Bridging the Gap Between Homogeneous and Heterogeneous Asynchronous Optimization Is Surprisingly Difficult

Read the original on arXiv Machine Learning →

The paper examines the challenge of bridging the performance gap between homogeneous and heterogeneous asynchronous optimization in large-scale machine learning. It demonstrates that under common first- and second-order similarity assumptions, no randomized algorithm can improve the pessimistic time complexity bounds for heterogeneous settings. The authors further show that even weak interpolation is insufficient, but by combining strong interpolation with a local Polyak‑Lojasiewicz condition, they achieve a new time complexity that matches the best-known homogeneous result without requiring identical data distributions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 14

High-Probability Convergence of SGD via Batched Updates

The paper introduces Batched SGD, a variant that groups online samples into epochs and performs a single update per epoch using a low‑variance gradient estimate. This batching approach allows a straightforward high‑probability analysis without restrictive assumptions or auxiliary sequences, yielding near‑optimal rates for both strongly convex and non‑convex objectives under standard smoothness and sub‑Gaussian noise conditions. The authors also extend the method to federated learning, providing the first high‑probability guarantees with logarithmic communication complexity, linear speedup in the number of agents, and robustness to data heterogeneity.

By Feng Zhu, Robert W. Heath Jr., Aritra Mitra
agents
More like this →