arXiv Machine Learning June 4, 2026 By Runxiong Wu, Andi Wang

On the Relationship Between CoCoA and ADMM for Distributed Empirical Risk Minimization

Read the original on arXiv Machine Learning →

arXiv:2502. 00470v3 Announce Type: replace-cross Abstract: Distributed empirical risk minimization (ERM) is often studied through two influential yet seemingly separate families of methods: CoCoA-type algorithms, derived from distributed dual coordinate ascent, and ADMM-type algorithms, derived from consensus and proximal splitting.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 20

Regularity-Aware Stochastic MGDA with Adaptive Conflict-Avoidant Update Direction Control

arXiv:2607. 15412v1 Announce Type: new Abstract: Multi-objective learning (MOL) aims to optimize multiple objectives simultaneously.

By Chentong Huang, Lisha Chen

safety

arXiv Machine Learning

Jun 25

A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization

arXiv:2602. 02877v2 Announce Type: replace Abstract: This paper studies optimization for a family of problems termed $\textbf{compositional entropic risk minimization}$, in which each data's loss is formulated as a Log-Expectation-Exponential (Log-E-Exp) function.

By Xiyuan Wei, Linli Zhou, Bokun Wang, Chih-Jen Lin, Tianbao Yang

arXiv AI

Jul 8

K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)

arXiv:2607. 05903v1 Announce Type: cross Abstract: We present K-ABENA (K-Adaptive Backpropagation with Error-based N-exclusion Algorithm), a selective gradient computation framework that reduces per-iteration training cost by excluding a fraction of low-loss ("minor") observations from the backward pass.

By Jean-Francois Bonbhel

safety

arXiv Machine Learning

Jun 29

Second-Order KKT Guarantees for Bregman ADMM in Nonconvex and Non-Lipschitz Optimization

arXiv:2606. 28307v1 Announce Type: cross Abstract: We analyze Bregman ADMM for nonconvex linearly constrained problems under two-sided relative smoothness, a condition that replaces the standard Lipschitz gradient assumption with a Hessian comparison relative to a Bregman kernel.

By Shuang Li, Zhihui Zhu, Qiuwei Li

arXiv Machine Learning

Jun 2

Robust Learning of a Group DRO Neuron

arXiv:2601. 18115v2 Announce Type: replace Abstract: We study the problem of learning a single neuron under standard squared loss in the presence of arbitrary label noise and group-level distributional shifts, for a broad family of covariate distributions.

By Guyang Cao, Shuyao Li, Sushrut Karmalkar, Jelena Diakonikolas

llms benchmarks

arXiv Machine Learning

Jul 17

What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity

arXiv:2607. 14731v1 Announce Type: new Abstract: Local SGD, also known as Federated Averaging, is a widely used distributed optimization algorithm.

By Kumar Kshitij Patel, Rustem Islamov, Sebastian U Stich, Aurelien Lucchi, Eduard Gorbunov, Lingxiao Wang

Related stories

Regularity-Aware Stochastic MGDA with Adaptive Conflict-Avoidant Update Direction Control

A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization

K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)

Second-Order KKT Guarantees for Bregman ADMM in Nonconvex and Non-Lipschitz Optimization

Robust Learning of a Group DRO Neuron

What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity