arXiv Machine Learning

RAPTOR: Role-Aware Private Training for Mixture-of-Experts

Hugging Face Trending Papers
Jun 1

IntraShuffler: A Privacy Preserving Framework for Heterogeneous DP Federated Learning

Heterogeneous Differential Privacy (HDP) in Federated Learning (FL) allows clients to select individual privacy budgets ($\varepsilon_i$) according to institutional policies and data sensitivity. In practice, many HDP-FL systems employ $\varepsilon$-aware server aggregation to improve model utility by re-weighting client updates according to their declared privacy budgets.

Hugging Face Trending Papers
Aug 10

DistMoE: Private-data Rehearsal-free Routing in Mixture-of-Experts for Distributed Instruction Tuning

Multimodal Large Language Models (MLLMs) have shown strong multimodal instruction-following ability, but adapting them to diverse visual-language domains typically assumes centralized data access and costly joint training. This is restrictive when data is distributed across private, domain-specific, or permission-limited clients.

arXiv Machine Learning
Aug 28

When Privacy Hurts Mergeability: Geometry-Aware Model Merging under Differential Privacy

The paper investigates how differential privacy (DP) affects the ability to merge independently fine‑tuned task models into a single multi‑task model. It identifies two geometric challenges—local sharpness and reference drift—that hinder mergeability of private models. To address these, the authors propose DP‑Merging, a framework that uses a sharpness‑aware objective and a reference‑based alignment regularizer to improve mergeability while maintaining DP guarantees.

By Jin Liu, Junkang Liu, Ning Xi, Yinbin Miao, Dawei Wei, Ke Cheng, Jianfeng Ma