arXiv Machine Learning

OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation

The paper introduces OP-Print, a unified framework for membership inference attacks on on‑policy self‑distillation models. It shows that the supervision signal in such distillation is sparse and structured, focusing on uncertainty‑sensitive correction sites, and exploits this property to construct attack signals for both gray‑box and black‑box threat models. Experiments demonstrate that OP‑Print outperforms baseline attacks, which remain near random guessing, across various models and distillation schemes.

arXiv AI

Jul 14

What Does It Mean to Break a Distillation Defense?

arXiv:2606. 25059v2 Announce Type: replace-cross Abstract: Black-box LLMs (accessible only via API) are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its outputs.

By Lena Libon, Pura Peetathawatchai, Michael Aerni, Daniel Paleka, Florian Tram\`er

llms efficiency

arXiv AI

Oct 7

Does On-Policy Distillation for Safety Pose Backdoor Risks?

arXiv:2610.07654v1 Announce Type: cross Abstract: On-policy distillation (OPD) has attracted growing attention as an effective way to transfer capabilities from teacher models to student models. Rece...

By Jian Luo, Kehan Qi, Qingqiao Hu, Meilong Xu, Jiacheng Qiu, Weimin Lyu, Jiawei Zhou, Chao Chen

llms efficiency multimodal safety

arXiv Machine Learning

Oct 8

RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents

arXiv:2610.06401v2 Announce Type: replace-cross Abstract: Tool-using language-model agents are vulnerable to indirect prompt injection because they must act on untrusted external content. Existing tr...

By Mohamed Dhouib, Clement Elliker, Alexi Canesse, Ma\"el Jenny, Lucas-Andrei Thil, Mahammed El Sharkawy, Sonia Vanier, Elie Bursztein

llms agents efficiency benchmarks

arXiv AI

Sep 30

OPFL: Optimistic Verification of Federated Learning via Empirical Boundary

arXiv:2609.37011v1 Announce Type: cross Abstract: Federated learning enables multiple clients to collaboratively train models without sharing their private data. However, the lack of visibility into...

By Hongxu Su, Jianzhu Yao, Xuechao Wang, Pramod Viswanath

robotics multimodal safety

Hugging Face Trending Papers

Jul 30

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

Federated fine-tuning of large language models (LLMs) enables collaborative training without exposing raw data. However, a recent attack, NeuroImprint [1] (arXiv:2606.

llms fine-tuning safety

arXiv AI

Jun 15

From Prompts to Responses: Dual-Sided Data Leakage and Defense in Split Large Language Models

arXiv:2606. 14210v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in privacy-sensitive domains, where users must balance the risk of data exposure through external APIs against the high computational cost of local deployment.

By Zixuan Gu, Xiaojun Ye, Yang Liu

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jul 31

TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement

arXiv:2607. 27940v1 Announce Type: new Abstract: Federated fine-tuning of large language models (LLMs) enables collaborative training without exposing raw data.

By Cheng Wei (Honor Device Co., Ltd., Shenzhen, China)

llms fine-tuning safety

arXiv AI

Jul 7

TIER: Trajectory-Invariant Explanation Regularization for Membership Privacy

arXiv:2607. 02903v1 Announce Type: cross Abstract: Explainability is central to building trustworthy AI, yet explanation interfaces can inadvertently provide adversaries with an expanded privacy-related attack surfaces.

By Varun Sharma, Kar Wai Fok, Vrizlynn L. L. Thing

safety

arXiv Machine Learning

Jul 14

Reference-Based Distillation Detection in LLMs

arXiv:2607. 09692v1 Announce Type: new Abstract: Model distillation -- training on outputs from stronger third-party models -- is widely used to boost performance, but raises concerns about unfair advantages and policy violations.

By Rajat Rawat, Sizhe Chen, Akshay Anand, Michael Duan, Bob Rotsted, Sewon Min

llms efficiency

arXiv AI

Jun 11

Robust Privacy: Inference-Stage Privacy through Certified Robustness

arXiv:2601. 17360v2 Announce Type: replace-cross Abstract: An adversary observing a model's released prediction can infer sensitive attributes of the queried input, or even reconstruct representatives of the model's training data.

By Jiankai Jin, Xiangzheng Zhang, Zhao Liu, Wenzhuo Xu, Dongdong Yang, Deyue Zhang, Quanchen Zou

efficiency multimodal safety

arXiv Machine Learning

Jun 2

Causal Evaluation of Membership Inference Attacks

arXiv:2602. 02819v4 Announce Type: replace Abstract: Membership Inference Attacks (MIAs) aim to distinguish training points (members) from unseen data (non-members), and are widely used to quantify memorization and assess privacy risks.

By Mathieu Even, Cl\'ement Berenfeld, Linus Bleistein, Tudor Cebere, Julie Josse, Aur\'elien Bellet

llms fine-tuning safety

arXiv Machine Learning

Jun 2

Profiling Privacy Preservation Against Gradient Inversion Attacks in Tabular Federated Learning

arXiv:2606. 00986v1 Announce Type: new Abstract: Federated learning (FL) enables multiple data holders to train machine learning models collaboratively without centralizing raw data, making it useful in privacy sensitive domains such as healthcare and institutional data sharing.

By Ivo Osterberg Nilsson, Maximilian Birr Engvall, Viktor Valadi, Teddy Lazebnik

llms benchmarks safety

Related stories