arXiv:2605.08992v2 Announce Type: replace Abstract: Federated learning (FL) is increasingly used to fine-tune foundation models (FMs) on distributed private data. The community largely assumes that l...
arXiv:2607. 28658v1 Announce Type: cross Abstract: Federated pre-training offers a way to train foundation models on private or distributed data without centralizing the underlying datasets.
The paper introduces FedRouter, a personalized federated learning framework that clusters tasks rather than clients to build specialized language models. It employs local and global clustering of adapters to associate them with specific tasks and includes an evaluation router that directs test samples to the most suitable adapter. Experiments on a multitask dataset show FedRouter outperforms existing methods, achieving up to 6.1% relative improvement under task interference and 136% under generalization scenarios.
ORDERS is a personalized federated learning configuration that combines a shared backbone, private residual adapters, and a classifier, using geometric weights based on descending update norms, feature alignment, and private‑parameter perturbations. In extensive experiments on CIFAR‑10 and Sent140, ORDERS achieves slightly higher mean client accuracy than baseline methods, with a 5.47% reduction in parameter payload. Ablation studies show limited, dataset‑dependent benefits from norm ranking and alignment, and no clear advantage from perturbations.
arXiv:2606. 10595v1 Announce Type: cross Abstract: Federated Learning (FL) has emerged as a promising solution for data hunger in centralized learning.
HE‑OFT is a cryptographically secure one‑shot federated fine‑tuning protocol that protects privacy by exchanging only one encrypted contribution and never revealing the trained model. Each client fine‑tunes a low‑rank adapter and a classifier head on a frozen public backbone, keeps the adapter, and uploads an encrypted head displacement that the server combines using multiparty CKKS without decryption. On four text classification tasks and one vision task, HE‑OFT achieves 61–79 % accuracy, retaining 85–96 % of the accuracy of a disclosed model, while a test‑time query takes 443.1–1713.1 s on one core or 56.1–255.1 s on a GPU, with level restoration reducing traffic from 1.6 GiB to 13.5 MiB.
arXiv:2606. 15625v1 Announce Type: new Abstract: The continuous scaling of large language models (LLMs) incurs prohibitive computational costs, making Mixture-of-Experts (MoE) a scalable alternative for efficient fine-tuning via sparse activation.
arXiv:2609.07192v1 Announce Type: cross Abstract: Asynchronous federated learning improves scalability by updating the global model from a server-side buffer of client updates as they arrive, rather...
arXiv:2610.31786v1 Announce Type: new Abstract: Federated unlearning (FU) aims to remove a designated target client's influence from a trained model without the prohibitive cost of retraining from sc...
FedReason introduces a large language model (LLM) that dynamically selects aggregation rules for federated learning by inspecting client model weights in a sandboxed environment. The LLM evaluates candidate rules on a small proxy dataset and submits a Python program that the server executes until the next call, defaulting to FedAvg if the output is invalid. Experiments on CIFAR-10 and CIFAR-100 with ResNet-18 under various heterogeneity and Byzantine scenarios show FedReason outperforming FedAvg by an average of 1.26 percentage points and matching the best server‑side rule on CIFAR‑100, while incurring modest API costs.
The paper investigates federated learning where each client’s data consists of unknown mixtures of distinct tasks, a scenario termed compound heterogeneity. It shows that when tasks share a common feature geometry, the optimal model for a mixed client is a convex combination of task‑specific models, motivating input‑dependent routing to specialized experts. The authors propose FedSEE, a method that recovers task experts via a convex program and achieves better performance than baselines, reducing negative transfer by 2.9 points overall and 3.7 points for the worst‑served quartile.
arXiv:2608. 12108v1 Announce Type: new Abstract: Federated learning (FL) enables collaborative model training across distributed clients while keeping data local.