arXiv Machine Learning

ParaBlock: Communication-Computation Parallel Block Coordinate Federated Learning for Large Language Models

arXiv:2511. 19959v2 Announce Type: replace Abstract: Federated learning (FL) has been extensively studied as a privacy-preserving training paradigm.

arXiv AI

Jun 18

Efficient Zeroth-Order Federated Finetuning of Language Models on Resource-Constrained Devices

arXiv:2502. 10239v3 Announce Type: replace-cross Abstract: Federated Learning (FL) is a promising paradigm for finetuning Large Language Models (LLMs) across distributed data sources while preserving data privacy.

By Mohamed Aboelenien Ahmed, Kilian Pfeiffer, Ramin Khalili, Heba Khdr, J\"org Henkel

llms fine-tuning safety

arXiv Machine Learning

Jun 9

Federated Large Language Models: Current Progress and Future Directions

arXiv:2409. 15723v3 Announce Type: replace Abstract: Large Language Models have achieved impressive performance across diverse applications, yet their training typically depends on centralized data collection, raising serious privacy and governance concerns.

By Yuhang Yao, Jianyi Zhang, Junda Wu, Chengkai Huang, Yu Xia, Tong Yu, Ruiyi Zhang, Sungchul Kim, Ryan Rossi, Ang Li, Lina Yao, Julian McAuley, Yiran Chen, Carlee Joe-Wong

llms agents fine-tuning safety

arXiv Machine Learning

Jun 3

DECA: Decentralizing Block-Wise Adam for Efficient LLM Full-Parameter Fine-Tuning on Non-IID Data

arXiv:2606. 03209v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) in privacy-sensitive and resource-constrained environments remains challenging.

By Yunsheng Yuan, Shaowei Li, Kai Wang, Zhongyuan Sun, Zheng Zhang, Kai Han, Jun Luo, Feng Li

llms fine-tuning safety

arXiv AI

2d ago

Federated Prompt Learning: A Unified Framework, Empirical Analysis, and Future Directions

arXiv:2608. 13844v1 Announce Type: cross Abstract: Large language models (LLMs) have become core components of cloud-based intelligent services in academia and industry, yet their training and deployment are hindered by high computational costs, data centralization, and privacy concerns.

By Qinglin Yang, Chen Qiu, Hongyuan Zhang, Pengdeng Li, Yuan Liu, Zhihong Tian

llms fine-tuning safety

arXiv Machine Learning

Jul 21

Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering

arXiv:2310. 16152v5 Announce Type: replace-cross Abstract: Federated learning (FL) has become a key component in various language modeling applications such as machine translation, next-word prediction, and medical record analysis.

By Md Rafi Ur Rashid, Vishnu Asutosh Dasu, Kang Gu, Najrin Sultana, Shagufta Mehnaz

llms nlp safety

arXiv AI

Jun 16

Beyond Weights and Gradients: A Taxonomy of Federated Learning Messages

arXiv:2606. 16891v1 Announce Type: cross Abstract: Federated Learning is rapidly evolving beyond the exchange of traditional model weights and gradients, yet existing definitions fail to capture the full scope of modern payloads like synthetic data and federated analytics.

By Alvaro Javier Vargas Guerrero, Xinguang Wang, Quang Manh Doan, Guy Nagels

safety

arXiv Machine Learning

Jul 24

Overcoming the Communication-Performance Tradeoff in LLM Pretraining

arXiv:2508. 15706v3 Announce Type: replace Abstract: Communication-efficient distributed training algorithms (e.

By Amir Sarfi, Benjamin Th\'erien, Joel Lidin, Eugene Belilovsky

llms efficiency

arXiv Machine Learning

Jun 9

Efficient Scaling of LLM Training with Flexible Context Parallelism

arXiv:2602. 21788v2 Announce Type: replace-cross Abstract: Scaling long-context capabilities is crucial for Large Language Models (LLMs).

By Yifan Niu, Han Xiao, Dongyi Liu, Wei Zhou, Jia Li

llms efficiency

arXiv Machine Learning

Jun 10

FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection

arXiv:2604. 24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments.

By Yutong He, Zhengyang Huang, Jiahe Geng, Kun Yuan

arXiv Machine Learning

Jun 16

Mixtures of Subspaces for Bandwidth Efficient Context Parallel Training

arXiv:2606. 16384v1 Announce Type: new Abstract: Pretraining language models with extended context windows enhances their ability to leverage rich information during generation.

By Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi, Gil Avraham, Violetta Shevchenko, Yan Zuo, Chamin Hewa Koneputugodage, Alexander Long

llms

arXiv Machine Learning

Jun 9

TAMUNA: Doubly Accelerated Distributed Optimization under Partial Participation

arXiv:2302. 09832v4 Announce Type: replace Abstract: In distributed optimization and federated learning, slow and costly communication between parallel devices and the central server constitutes the primary bottleneck.

By Laurent Condat, Ivan Agarsk\'y, Grigory Malinovsky, Peter Richt\'arik

arXiv Machine Learning

Jun 25

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

arXiv:2606. 25627v1 Announce Type: new Abstract: Distributed intelligent systems increasingly need to train across data silos without centralizing raw data.

By Erdenebileg Batbaatar, Young Yoon

fine-tuning safety

Related stories