The paper investigates the use of Johnson‑Lindenstrauss (JL) transforms in Byzantine‑robust distributed optimization. By leveraging JL’s ability to preserve inner products and distances, worker messages are weighted based on their alignment with a trusted reference gradient, reducing communication costs while maintaining robustness. Experimental results support the theoretical claims.
The growing scale of data and models in machine learning calls for efficient optimization methods. Distributed optimization addresses this challenge, with compression mechanisms helping to reduce comm...
arXiv:2609. 11712v1 Announce Type: cross Abstract: In this paper, we investigate the generalization performance of distributed gradient descent algorithms in a reproducing kernel Hilbert space under a robust loss function $l_{\sigma}$.
arXiv:2605. 28335v2 Announce Type: replace Abstract: Federated Learning (FL) enables multiple clients to collaboratively train models without sharing raw data, but it is highly vulnerable to Byzantine attacks.
arXiv:2608. 06563v1 Announce Type: new Abstract: Machine learning and optimization have advanced together, with practical demands motivating new theory and theoretical breakthroughs enabling new applications.
arXiv:2607. 10970v1 Announce Type: new Abstract: Federated learning distributes data among $n$ clients, making it vulnerable to malicious attacks and data heterogeneity, which together pose challenges for robust learning.
Federated learning distributes data among $n$ clients, making it vulnerable to malicious attacks and data heterogeneity, which together pose challenges for robust learning. To tackle this issue, centered clipping and Huber aggregators have been exploited for Byzantine robustness.
arXiv:2412. 07151v1 Announce Type: cross Abstract: Distributed model training needs to be adapted to challenges such as the straggler effect and Byzantine attacks.
FedFit introduces a federated fine‑tuning framework for large language models that reduces communication overhead by using a disjoint shared vector‑bank parameterization to reconstruct adapter matrices from two compact global vector banks. It resolves the aggregation dilemma between Sum‑of‑Products and Product‑of‑Sums through an alternating optimization schedule that alternates between accurate single‑bank updates and joint updates corrected by a Residual Spectral Aggregation mechanism. The method also incorporates blockwise quantization with client‑side error feedback and provides theoretical convergence guarantees, achieving perplexity comparable to standard federated LoRA while delivering up to 100× higher compression ratios on Qwen2.5 models.
arXiv:2512.03579v2 Announce Type: replace Abstract: Optimal transport (OT) and Gromov-Wasserstein (GW) alignment provide interpretable geometric frameworks for comparing, transforming, and aggregatin...
The paper proposes Byrd-NAFL, a Byzantine‑robust federated learning algorithm that incorporates Nesterov’s momentum and resilient aggregation rules. It achieves fast and safe convergence under non‑convex, smooth loss functions with relaxed gradient assumptions, and provides a finite‑time convergence guarantee. Experiments show that Byrd-NAFL outperforms existing methods in convergence speed, accuracy, and resilience to various malicious attacks.
The paper investigates how well two-stage structured Hadamard rotations approximate uniform random rotations in high-dimensional spaces. It establishes nonasymptotic upper bounds of order √{log d/d} for one-dimensional marginals, while showing a persistent Wasserstein-1 discrepancy for the full vector that does not vanish as dimension grows. These results highlight a separation between marginal and joint approximation quality, though they do not directly translate into algorithmic performance guarantees.
arXiv:2602. 04396v2 Announce Type: replace-cross Abstract: Distributed training of foundation models via $\texttt{DDP}$ is limited by interconnect bandwidth.