← Back to all news
arXiv Machine Learning By Yunsheng Yuan, Shaowei Li, Kai Wang, Zhongyuan Sun, Zheng Zhang, Kai Han, Jun Luo, Feng Li

DECA: Decentralizing Block-Wise Adam for Efficient LLM Full-Parameter Fine-Tuning on Non-IID Data

Read the original on arXiv Machine Learning →

arXiv:2606. 03209v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) in privacy-sensitive and resource-constrained environments remains challenging.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Jun 16

SDFLoRA: Selective Decoupled Federated LoRA for Privacy-preserving Fine-tuning with Heterogeneous Clients

arXiv:2601. 11219v3 Announce Type: replace-cross Abstract: Federated learning (FL) for large language models (LLMs) has attracted increasing attention as a privacy-preserving approach for adapting models over distributed data, where parameter-efficient methods such as Low-Rank Adaptation (LoRA) are widely adopted to reduce communication and memory costs.

By Zhikang Shen, Jianrong Lu, Haiyuan Wan, Jianhai Chen
llmsfine-tuningbenchmarkssafety
More like this →
arXiv AI
Sep 15

FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks

FLoKD is an adaptive knowledge‑distillation framework designed for federated fine‑tuning of low‑rank LLMs over wireless networks. It transmits intermediate LoRA activations instead of full parameters or token‑level logits, and uses transformer block importance scoring plus dataset selection to reduce communication. Experiments on WikiText‑103, PTB, and Dialog show a 50‑65% reduction in communication while maintaining competitive perplexity.

By Xinlu Zhang, Na Yan, Yang Su, Yansha Deng, Toktam Mahmoodi
llmsnlpfine-tuningefficiencysafety
More like this →
arXiv Machine Learning
Sep 2

FlexP-SFT: A Flexible Aggregation-Free Framework for On-Device Personalized Split Federated Fine-Tuning of LLMs

FlexP-SFT introduces an aggregation-free framework for personalized split federated fine-tuning of large language models, eliminating the client-side aggregation step that traditionally causes communication bottlenecks and straggler issues. The method employs a layer‑flexible alignment strategy to balance personalization and generalization without global synchronization, and formulates split‑ratio selection as a resource‑aware discrete optimization problem. Experiments demonstrate that FlexP-SFT improves both accuracy and latency compared to baselines, achieving a superior resource‑accuracy trade‑off.

By Jiaxiang Geng, Tianjun Yuan, Pengchao Han, Ying Gao, Xianhao Chen, Bing Luo
llmsfine-tuningsafety
More like this →