arXiv Machine Learning July 2, 2026 By Radhakrishna Achanta, Will Reed

TallyTrain: Communication-Efficient Federated Distillation

Read the original on arXiv Machine Learning →

arXiv:2607. 00173v1 Announce Type: new Abstract: Federated learning is bandwidth-bound on two orthogonal axes: model size, which limits how often parameter-averaging methods can afford to merge, and class count, which makes per-probe soft-label distillation prohibitive at large vocabularies.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 16

Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning

arXiv:2512. 12737v2 Announce Type: replace Abstract: Decentralized federated learning (DFL) enables collaborative model training without a central server, but converges slowly under statistical heterogeneity.

By Li Xia

data-engineering

arXiv AI

Aug 11

FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

arXiv:2608. 09221v1 Announce Type: cross Abstract: Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy.

By Radwan Selo, Majid Kundroo, Taehong Kim

benchmarks safety data-engineering

arXiv AI

1d ago

FedImp: Enhancing Federated Learning Convergence with Impurity-Based Weighting

arXiv:2608. 14654v1 Announce Type: cross Abstract: Federated Learning (FL) is a collaborative paradigm that enables multiple devices to train a global model while preserving local data privacy.

By Hai Anh Tran, Cuong Ta, Truong X. Tran

safety

Hugging Face Trending Papers

Aug 10

FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in dataset sizes, which can lead to biased model updates and hinder convergence.

benchmarks safety data-engineering

arXiv Machine Learning

Jun 5

Unifying Dataset Pruning and Distillation for Efficient Large-scale Compression

arXiv:2502. 06434v2 Announce Type: replace-cross Abstract: Dataset pruning (DP) and dataset distillation (DD) fundamentally differ in their outputs: DP selects original image subsets, while DD generates synthetic images.

By Lingao Xiao, Songhua Liu, Yang He, Xinchao Wang

efficiency benchmarks

arXiv Machine Learning

Jun 16

Photon: Federated LLM Pre-Training

arXiv:2411. 02908v2 Announce Type: replace Abstract: Scaling large language models (LLMs) demands extensive data and computing resources, which are traditionally constrained to data centers by the high-bandwidth requirements of distributed training.

By Lorenzo Sani, Alex Iacob, Zeyu Cao, Royson Lee, Bill Marino, Yan Gao, Dongqi Cai, Zexi Li, Wanru Zhao, Xinchi Qiu, Nicholas D. Lane

llms

Related stories

Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learning

FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

FedImp: Enhancing Federated Learning Convergence with Impurity-Based Weighting

FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

Unifying Dataset Pruning and Distillation for Efficient Large-scale Compression

Photon: Federated LLM Pre-Training