Correcting Before Adapting: Gradient Correction for Federated Adaptive Optimization

The paper introduces Federated Adaptive Gradient Correction (FA‑GC), a method that corrects each gradient before it is incorporated into the running averages used by adaptive optimizers like Adam in federated learning. Compared to a naive correction approach (FA‑NT) that only adjusts the final update direction, FA‑GC ensures that the corrected direction only vanishes at global stationary points and provides convergence guarantees without client‑heterogeneity terms. Experiments on seven image and language tasks show that FA‑GC with Adam achieves target accuracy in fewer rounds and, in some cases, reduces total communication compared to uncorrected local Adam and FA‑NT.