arXiv Machine Learning

First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints

arXiv:2603. 05774v2 Announce Type: replace Abstract: This paper addresses the distributed stochastic minimax optimization problem subject to stochastic constraints.

arXiv Machine Learning

Aug 10

Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization

arXiv:2608. 06563v1 Announce Type: new Abstract: Machine learning and optimization have advanced together, with practical demands motivating new theory and theoretical breakthroughs enabling new applications.

By Grigory Malinovsky

fine-tuning

arXiv AI

Jul 2

Entropy-Regularized Probabilistic Gates for Sparse Model Discovery in Scarce-Data Federated Learning

arXiv:2607. 00275v1 Announce Type: cross Abstract: Federated Learning (FL) is a distributed machine learning (ML) paradigm with collaboration among multiple clients without sharing data.

By Krishna Harsha Kovelakuntla Huthasana, Alireza Olama, Andreas Lundell

efficiency benchmarks

arXiv Machine Learning

Jun 3

Online Learning with Gradient-Variation Interval Regret

arXiv:2606. 03831v1 Announce Type: new Abstract: This paper investigates non-stationary online learning using the metric of interval regret, which requires an online algorithm to perform well over every time interval.

By Yan-Feng Xie, Shuche Wang, Peng Zhao, Zhi-Hua Zhou

safety

arXiv Machine Learning

Jun 10

FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection

arXiv:2604. 24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments.

By Yutong He, Zhengyang Huang, Jiahe Geng, Kun Yuan

arXiv Machine Learning

Jul 17

What's in a Smoothness Constant? Tighter Rates for Local SGD with Bounded Second-order Heterogeneity

arXiv:2607. 14731v1 Announce Type: new Abstract: Local SGD, also known as Federated Averaging, is a widely used distributed optimization algorithm.

By Kumar Kshitij Patel, Rustem Islamov, Sebastian U Stich, Aurelien Lucchi, Eduard Gorbunov, Lingxiao Wang

arXiv Machine Learning

Jul 17

Primal-dual algorithm for contextual stochastic combinatorial optimization

arXiv:2505. 04757v2 Announce Type: replace Abstract: This paper introduces a novel approach to contextual stochastic optimization, integrating operations research and machine learning to address decision-making under uncertainty.

By Louis Bouvier, Thibault Prunet, Vincent Lecl\`ere, Axel Parmentier

benchmarks

arXiv Machine Learning

Jul 1

Constrained Online Convex Optimization without Slater's Condition

arXiv:2606. 31480v1 Announce Type: new Abstract: We study constrained online convex optimization with adversarial losses and stochastic or adversarial constraints.

By Kihyun Yu, Junehee Lee, Dabeen Lee

safety

arXiv Machine Learning

Jul 15

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication

arXiv:2405. 11667v2 Announce Type: replace Abstract: Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD.

By Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari, Lingxiao Wang, Sebastian U. Stich, Ziheng Cheng, Nirmit Joshi, Nathan Srebro

arXiv Machine Learning

Jun 2

Dimensionality Reduction for Robust Federated Learning: A Theoretical Analysis and Convergence Guarantee

arXiv:2605. 28335v2 Announce Type: replace Abstract: Federated Learning (FL) enables multiple clients to collaboratively train models without sharing raw data, but it is highly vulnerable to Byzantine attacks.

By Shiyuan Zuo, Jiashuo Li, Rongfei Fan, Han Hu, Jie Xu

rag benchmarks

arXiv Machine Learning

Jun 24

Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum

arXiv:2406. 13041v3 Announce Type: replace Abstract: Lower-bound analyses for nonconvex strongly-concave minimax optimization problems have shown that stochastic first-order algorithms require at least $\mathcal{O}(\varepsilon^{-4})$ sample complexity to find an $\varepsilon$-stationary point.

By Haoyuan Cai, Sulaiman A. Alghunaim, Ali H. Sayed

safety

arXiv Machine Learning

Jun 8

Residual-Controlled Multiplier Learning for Stochastic Constrained Decision-Making

arXiv:2606. 07088v1 Announce Type: new Abstract: Stochastic constrained decision-making requires optimizing performance objectives while enforcing statistical requirements such as safety or fairness.

By Kang Liu, Jianchen Hu, Ziyu Qu

safety

arXiv Machine Learning

Jul 3

Class-Grouped Normalized Momentum and Faster Hyperparameter Exploration to Tackle Class Imbalance in Federated Learning

arXiv:2607. 01474v1 Announce Type: new Abstract: Class imbalance poses a critical challenge in federated learning (FL), where underrepresented classes suffer from poor predictive performance yet cannot be addressed by standard centralized techniques due to privacy and heterogeneity constraints.

By Haemin Park, Diego Klabjan, Martin W. Braun, Xiuqi Li, Balakrishnan Ananthanarayanan

reinforcement-learning benchmarks safety

Related stories