CALM introduces a smooth trust gating mechanism for decentralized federated learning, replacing hard filtering of teacher models with class‑wise, sample‑wise, and label‑based weighting. It allows clients with heterogeneous architectures to distill knowledge from peers without a central server or shared data, even under severe non‑IID label skew. Experiments on CIFAR‑10, SVHN, OrganAMNIST, and Google Speech Commands show that CALM consistently outperforms uniform and hard‑filtered distillation and matches or exceeds other heterogeneous‑FL methods.
arXiv:2609.00446v1 Announce Type: new Abstract: Conventional federated learning (FL) relies on parameter averaging, which forces clients to be doubly homogeneous: it demands an identical architecture...
arXiv:2607. 00173v1 Announce Type: new Abstract: Federated learning is bandwidth-bound on two orthogonal axes: model size, which limits how often parameter-averaging methods can afford to merge, and class count, which makes per-probe soft-label distillation prohibitive at large vocabularies.
arXiv:2609.23697v1 Announce Type: cross Abstract: Multi-teacher on-policy distillation allows a student to learn from complementary specialists on its own trajectories. Domain-routed approaches, howe...
arXiv:2608. 02250v1 Announce Type: new Abstract: Federated learning (FL) is a popular distributed learning framework where multiple clients perform local training and a server aggregates the locally updated models.
arXiv:2609.36546v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student model on its self-generated trajectories with dense token-level teacher feedback. However, naive OPD ma...
The paper introduces a robust decentralized federated distillation approach that allows heterogeneous client models to collaborate using predictions on shared unlabeled public data. Each client evaluates received predictions across three modalities—class prediction, boundary decision, and prediction correlation—filters unreliable clients, assigns reliability-based weights, and constructs modality-specific teachers. The method validates distillation gradients against supervised gradients from private data, removes conflicting gradients, and proves convergence under Byzantine attacks, achieving improved accuracy on CIFAR-10 and CIFAR-100 under non‑IID data and malicious conditions.
arXiv:2605.08992v2 Announce Type: replace Abstract: Federated learning (FL) is increasingly used to fine-tune foundation models (FMs) on distributed private data. The community largely assumes that l...
Asynchronous federated learning improves scalability by updating the global model from a server-side buffer of client updates as they arrive, rather than waiting for all selected clients to finish. Wh...
The paper presents a practical approach to semi‑supervised federated learning for automatic speech recognition (ASR). It demonstrates that using a per‑client online teacher combined with a stabilizing server‑side anchor—where the server continues training on labeled data between rounds—significantly reduces divergence caused by pseudo‑label errors. The authors provide design guidelines that improve in‑domain performance by an average of 20.8 % and cross‑domain performance by 10.0 % over the best prior method, narrowing the gap to fully‑supervised federated learning.
The paper introduces Teacher-Gated On-Policy Distillation (TGOPD), a method that verifies teacher reliability at the prompt level before applying dense supervision in on-policy distillation. TGOPD uses verifier-scored teacher probes to decide whether to route a prompt to dense OPD or to a verifier-grounded alternative. Experiments on 4B and 35B models across mathematics, code, and instruction tasks show TGOPD outperforms vanilla OPD and improves teacher GPU utilization from 9.8% to 78.9% in a 4B single-domain run.
arXiv:2609.36660v1 Announce Type: new Abstract: We study federated learning (FL) with adversarial clients, where the goal is to minimize the average loss of the honest (non-adversarial) clients witho...
arXiv:2609.07192v1 Announce Type: cross Abstract: Asynchronous federated learning improves scalability by updating the global model from a server-side buffer of client updates as they arrive, rather...