arXiv Machine Learning June 16, 2026 By Yijun Lu, Zihan Fang, Pengpeng Qiao, Zheng Lin, Jing Yang, Yuxin Zhang, Por Lip Yee, Zhe Chen, Jun Luo

Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts

Read the original on arXiv Machine Learning →

arXiv:2606. 15625v1 Announce Type: new Abstract: The continuous scaling of large language models (LLMs) incurs prohibitive computational costs, making Mixture-of-Experts (MoE) a scalable alternative for efficient fine-tuning via sparse activation.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

6d ago

Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning

arXiv:2608. 12108v1 Announce Type: new Abstract: Federated learning (FL) enables collaborative model training across distributed clients while keeping data local.

By Mirko Konstantin, Stefan Zachow, Anirban Mukhopadhyay

arXiv Machine Learning

1d ago

Global Federated Learning Strategies for Building Efficient Personalized Models

arXiv:2608. 15107v1 Announce Type: new Abstract: Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously.

By Seongyoon Kim

reinforcement-learning fine-tuning efficiency safety

Hugging Face Trending Papers

Jul 29

FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA

Federated PEFT enables LLMs to collaboratively adapt to decentralized private data without sharing raw examples. However, task heterogeneity across clients can cause cross-task interference and gradient conflicts during aggregation.

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 30

FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA

arXiv:2607. 26618v1 Announce Type: new Abstract: Federated PEFT enables LLMs to collaboratively adapt to decentralized private data without sharing raw examples.

By Donghang Duan, Xu Zheng, Lizong Zhang, Chong Mu, Meng Han

llms fine-tuning benchmarks

arXiv AI

Jun 16

Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments

arXiv:2505. 19699v2 Announce Type: replace-cross Abstract: Federated Learning (FL) is a decentralized machine learning paradigm that enables clients to collaboratively train models while preserving data privacy.

By Junming Liu, Yanting Gao, Yuqi Li, Siyuan Meng, Yifei Sun, Aoqi Wu, Yirong Chen, Ding Wang, Shiping Wen

efficiency multimodal benchmarks safety

arXiv AI

Jun 8

Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning

arXiv:2606. 07500v1 Announce Type: cross Abstract: Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge.

By Fatema Siddika, Md Anwar Hossen, Tanwi Mallick, Ali Jannesari

llms benchmarks

Related stories

Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning

Global Federated Learning Strategies for Building Efficient Personalized Models

FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA

FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA

Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments

Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning