arXiv Computation and Language September 11, 2026 By Jordi Luque, Fernando L\'opez, Aleix Sant

Component-Aware Differential Privacy for Federated Multilingual Speech-LLMs

Read the original on arXiv Computation and Language →

The paper introduces a per-layer differential privacy (DP) clipping strategy for federated multilingual speech large language models (speech‑LLMs). It demonstrates that standard single‑pool per‑layer DP methods fail due to a cross‑component budget collapse caused by large norm differences between acoustic encoders and language decoders. The authors propose an α‑split two‑pool allocation that normalises encoder and decoder parameters separately, preserving the overall DP guarantee while restoring word error rate performance and providing tighter noise protection for the encoder.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

2d ago

Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?

arXiv:2609.40335v1 Announce Type: new Abstract: Differentially Private Stochastic Gradient Descent (DP-SGD) is a leading approach for privacy-preserving fine-tuning of large language models (LLMs). M...

By Razan El Mais, Ali Chehab, Ibrahim Issa, Razane Tajeddine

llms rag fine-tuning safety

arXiv Machine Learning

Aug 5

Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models

arXiv:2608. 03277v1 Announce Type: new Abstract: Differentially private zeroth-order optimization (DP-ZO) enables memory-efficient private fine-tuning of large language models using only forward evaluations.

By Lele Zheng, Weifeng Kong, Xinyi Zhang, Ke Cheng, Tao Zhang, Yulong Shen

llms fine-tuning safety

arXiv Machine Learning

Jul 31

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

arXiv:2607. 27940v1 Announce Type: new Abstract: Federated fine-tuning of large language models (LLMs) enables collaborative training without exposing raw data.

By Cheng Wei (Honor Device Co., Ltd., Shenzhen, China)

llms fine-tuning safety

Hugging Face Trending Papers

Jul 30

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

Federated fine-tuning of large language models (LLMs) enables collaborative training without exposing raw data. However, a recent attack, NeuroImprint [1] (arXiv:2606.

llms fine-tuning safety

arXiv Machine Learning

Aug 18

An Adaptive Gradient Clipping and Noise Injection Mechanism for Differentially Private Federated Learning

arXiv:2608. 15153v1 Announce Type: cross Abstract: Differentially private federated learning must balance privacy protection against model accuracy and training efficiency.

By Wenjing Wei, Alla Jammine, Farid Nait-Abdesselam

safety

arXiv Machine Learning

Jul 23

End-to-End Differential Privacy in Training Deep Neural Network Classifiers

arXiv:2607. 19580v1 Announce Type: new Abstract: Differentially private machine learning enables model training on sensitive data while ensuring that individual data is unlikely to be recoverable from the parameters of the resulting model.

By Huaiyuan Rao, Calvin Hawkins, Alexander Benvenuti, Matthew Hale

benchmarks safety

One email a morning, machine-written

Related stories

Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?

Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

An Adaptive Gradient Clipping and Noise Injection Mechanism for Differentially Private Federated Learning

End-to-End Differential Privacy in Training Deep Neural Network Classifiers