arXiv Machine Learning September 7, 2026 By Georgios Politis, Evangelos Pappas

Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys

Read the original on arXiv Machine Learning →

The paper reports a privacy breach in a two-node split‑LLM training system where the returned gradient reveals which data rows were real, despite the system passing standard privacy checks. By exploiting the fact that decoy rows produce zero gradients, an attacker can identify real rows with 100% accuracy across multiple runs. The authors demonstrate that adding gradient clipping and noise can mitigate the leak, but the system remains vulnerable to several untested attack vectors.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 31

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

arXiv:2607. 27940v1 Announce Type: new Abstract: Federated fine-tuning of large language models (LLMs) enables collaborative training without exposing raw data.

By Cheng Wei (Honor Device Co., Ltd., Shenzhen, China)

llms fine-tuning safety

arXiv AI

Jun 16

AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems

arXiv:2602. 11510v3 Announce Type: replace Abstract: Multi-agent Large Language Model (LLM) systems create privacy risks that current output-only benchmarks cannot measure.

By Faouzi El Yagoubi, Godwin Badu-Marfo, Ranwa Al Mallah

llms agents benchmarks safety

arXiv Machine Learning

Jun 17

Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs

arXiv:2606. 17110v1 Announce Type: cross Abstract: Large Language Models are increasingly trained on proprietary or sensitive data, from private healthcare and financial records to user conversations containing secrets.

By Md Abdullah Al Mamun, Ngoc Phu Doan, Pedram Zaree, Ihsen Alouani, Nael Abu-Ghazaleh

llms multimodal safety

Hugging Face Trending Papers

Jul 30

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

Federated fine-tuning of large language models (LLMs) enables collaborative training without exposing raw data. However, a recent attack, NeuroImprint [1] (arXiv:2606.

llms fine-tuning safety

arXiv AI

Jun 15

From Prompts to Responses: Dual-Sided Data Leakage and Defense in Split Large Language Models

arXiv:2606. 14210v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in privacy-sensitive domains, where users must balance the risk of data exposure through external APIs against the high computational cost of local deployment.

By Zixuan Gu, Xiaojun Ye, Yang Liu

llms fine-tuning benchmarks safety

arXiv AI

Sep 10

PAC-Private Autoregressive Generation: Calibrating Noise to Ensemble Disagreement

The paper introduces PAC‑Private Autoregressive Generation, a method that calibrates noise based on ensemble disagreement across overlapping ‘worlds’ of a private corpus, thereby extending PAC privacy from classification to text generation. By training adapters on a frozen public model and using posterior‑weighted disagreement to add noise only when predictions vary, the approach achieves strong privacy guarantees while preserving most of the fine‑tuning benefit. Experiments on WikiText‑103 with GPT‑2‑small show 74 % of the fine‑tuning gain retained with a per‑token budget of 2⁻³², and membership‑inference success bounded to 51.08 % after one million tokens, outperforming PMixED under matched conditions.

By Mina Mirzadehsarcheshmeh, Amir Keyvan Khandani

llms fine-tuning safety

One email a morning, machine-written

Related stories

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems

Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

From Prompts to Responses: Dual-Sided Data Leakage and Defense in Split Large Language Models

PAC-Private Autoregressive Generation: Calibrating Noise to Ensemble Disagreement