The paper investigates the use of Johnson‑Lindenstrauss (JL) transforms in Byzantine‑robust distributed optimization. By leveraging JL’s ability to preserve inner products and distances, worker messages are weighted based on their alignment with a trusted reference gradient, reducing communication costs while maintaining robustness. Experimental results support the theoretical claims.
arXiv:2609. 11712v1 Announce Type: cross Abstract: In this paper, we investigate the generalization performance of distributed gradient descent algorithms in a reproducing kernel Hilbert space under a robust loss function $l_{\sigma}$.
arXiv:2605. 28335v2 Announce Type: replace Abstract: Federated Learning (FL) enables multiple clients to collaboratively train models without sharing raw data, but it is highly vulnerable to Byzantine attacks.
arXiv:2608. 06563v1 Announce Type: new Abstract: Machine learning and optimization have advanced together, with practical demands motivating new theory and theoretical breakthroughs enabling new applications.
Federated learning distributes data among $n$ clients, making it vulnerable to malicious attacks and data heterogeneity, which together pose challenges for robust learning. To tackle this issue, centered clipping and Huber aggregators have been exploited for Byzantine robustness.
FedFit introduces a federated fine‑tuning framework for large language models that reduces communication overhead by using a disjoint shared vector‑bank parameterization to reconstruct adapter matrices from two compact global vector banks. It resolves the aggregation dilemma between Sum‑of‑Products and Product‑of‑Sums through an alternating optimization schedule that alternates between accurate single‑bank updates and joint updates corrected by a Residual Spectral Aggregation mechanism. The method also incorporates blockwise quantization with client‑side error feedback and provides theoretical convergence guarantees, achieving perplexity comparable to standard federated LoRA while delivering up to 100× higher compression ratios on Qwen2.5 models.
arXiv:2607. 10970v1 Announce Type: new Abstract: Federated learning distributes data among $n$ clients, making it vulnerable to malicious attacks and data heterogeneity, which together pose challenges for robust learning.
arXiv:2512.03579v2 Announce Type: replace Abstract: Optimal transport (OT) and Gromov-Wasserstein (GW) alignment provide interpretable geometric frameworks for comparing, transforming, and aggregatin...
arXiv:2606. 10916v1 Announce Type: cross Abstract: This paper introduces range regularization for federated learning with linear systematic components to enhance statistical accuracy and induce cross-client regularity conducive to quantization, coding, and resource efficiency.
The paper investigates how well two-stage structured Hadamard rotations approximate uniform random rotations in high-dimensional spaces. It establishes nonasymptotic upper bounds of order √{log d/d} for one-dimensional marginals, while showing a persistent Wasserstein-1 discrepancy for the full vector that does not vanish as dimension grows. These results highlight a separation between marginal and joint approximation quality, though they do not directly translate into algorithmic performance guarantees.
arXiv:2412. 07151v1 Announce Type: cross Abstract: Distributed model training needs to be adapted to challenges such as the straggler effect and Byzantine attacks.
Geodesic‑SVDD introduces a distributed learning framework for structured tabular data that transforms each client’s task into a one‑class density estimation on a hypersphere, enabling aggregation of local models as probabilistic manifolds in a single communication round. The method employs a Residual Spherical Encoder and a Geodesic Latent Mixup to enforce strict hyperspherical constraints, preventing manifold collapse, and proposes a One‑Shot Product‑of‑Experts protocol for privacy‑preserving aggregation without iterative synchronization. Experiments on five tabular benchmarks and varying label skew levels show that Geodesic‑SVDD achieves the highest mean AUROC among seven federated methods, surpasses federated tree ensembles by up to 21 AUROC points, and its One‑Shot variant matches centralized Deep SVDD under IID and mild skew conditions.