arXiv AI

HE-Guardrail: A Homomorphic Guardrail Against Jailbreak Attacks for Encrypted Large Language Model Inference

HE-Guardrail is a framework that applies homomorphic encryption to enforce guardrails against jailbreak attacks during encrypted large language model inference. It evaluates guardrail mechanisms—Llama Guard, JBShield, and GradSafe—directly on encrypted data, deciding whether to return the model’s response to the client. The approach preserves confidentiality while closely matching the decisions of plaintext guardrails, offering varied security-efficiency-utility trade‑offs.

arXiv AI

Sep 11

Arbitrary Cipher Attacks Against Large Language Models Do Not Require Fine-Tuning

The paper reports that large language models can acquire cipher-based covert communication skills without fine‑tuning, using prompting or in‑context learning instead. This enables new jailbreak attacks that bypass alignment safeguards by encrypting harmful requests, making them appear as nonsensical text to harmfulness classifiers. The authors demonstrate successful attacks against frontier models from Anthropic, Google, and OpenAI.

By Thomas Rivasseau

llms fine-tuning safety

arXiv AI

Aug 11

Stealing Reasoning Traces from Proprietary LLM APIs

arXiv:2608. 09867v1 Announce Type: cross Abstract: Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage.

By Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner, Joachim Schaeffer, Ameya Prabhu, Jonas Geiping, Maksym Andriushchenko

llms rag agents efficiency safety

arXiv AI

Jun 15

From Prompts to Responses: Dual-Sided Data Leakage and Defense in Split Large Language Models

arXiv:2606. 14210v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in privacy-sensitive domains, where users must balance the risk of data exposure through external APIs against the high computational cost of local deployment.

By Zixuan Gu, Xiaojun Ye, Yang Liu

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jul 21

Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering

arXiv:2310. 16152v5 Announce Type: replace-cross Abstract: Federated learning (FL) has become a key component in various language modeling applications such as machine translation, next-word prediction, and medical record analysis.

By Md Rafi Ur Rashid, Vishnu Asutosh Dasu, Kang Gu, Najrin Sultana, Shagufta Mehnaz

llms nlp safety

arXiv AI

Jul 9

PRoVeFL: Private Robust and Verifiable Aggregation in Federated Learning

arXiv:2607. 06612v1 Announce Type: cross Abstract: Federated Learning (FL) enables multiple clients to collaboratively train machine learning models while retaining data locality, thereby enhancing user privacy.

By Harsh Kasyap, Anil Kumar Pradhan, Ugur Ilker Atmaca, Graham Cormode, Carsten Maple

benchmarks safety

Hugging Face Trending Papers

Jul 7

PRoVeFL: Private Robust and Verifiable Aggregation in Federated Learning

Federated Learning (FL) enables multiple clients to collaboratively train machine learning models while retaining data locality, thereby enhancing user privacy. However, traditional FL frameworks rely on a centralized aggregation server and assume honest-but-curious clients, making them susceptible to both server-side inference and client-side poisoning attacks.

benchmarks safety

arXiv Machine Learning

Jun 2

VERA: Variational Inference Framework for Jailbreaking Large Language Models

arXiv:2506. 22666v3 Announce Type: replace-cross Abstract: The rise of API-only access to state-of-the-art LLMs highlights the need for effective black-box jailbreak methods to identify model vulnerabilities in real-world settings.

By Anamika Lochab, Lu Yan, Patrick Pynadath, Xiangyu Zhang, Ruqi Zhang

llms benchmarks safety

arXiv Machine Learning

Jul 20

Code-Poisoning Property Inference Attacks

arXiv:2607. 15970v1 Announce Type: cross Abstract: The flourishing code hosting platforms and coding agents enable even beginners with private data to build tailored Machine Learning (ML) models using available code quickly.

By Xukun Luan, Yuhui Gong, Gang Zhang, Zixuan Huang, Yuanguo Bi, Xuesong Li, Jinyan Liu

agents safety

arXiv Machine Learning

Jun 4

Bypassing Prompt Guards in Production with Controlled-Release Prompting

arXiv:2510. 01529v3 Announce Type: replace Abstract: Ball et al.

By Jaiden Fairoze, Sanjam Garg, Keewoo Lee, Mingyuan Wang

llms safety

arXiv Machine Learning

Jun 17

Loss Landscape Poisoning: Targeted Extraction of Unseen Training Data from LLMs

arXiv:2606. 17110v1 Announce Type: cross Abstract: Large Language Models are increasingly trained on proprietary or sensitive data, from private healthcare and financial records to user conversations containing secrets.

By Md Abdullah Al Mamun, Ngoc Phu Doan, Pedram Zaree, Ihsen Alouani, Nael Abu-Ghazaleh

llms multimodal safety

arXiv Machine Learning

Jun 30

FlipGuard: Defending Large Language Models Against Quantization-Conditioned Backdoor Attacks

arXiv:2606. 28962v1 Announce Type: cross Abstract: Model quantization is essential for the efficient deployment of Large Language Models (LLMs), but introduces a critical vulnerability: Quantization-Conditioned Backdoor (QCB) attacks.

By Aoying Zheng, Anqi Du, Zizhuang Deng, Yuxuan Chen

llms efficiency safety

arXiv AI

Jun 2

Safe-FedLLM: Delving into the Safety of Federated Large Language Models

arXiv:2601. 07177v5 Announce Type: replace-cross Abstract: Federated learning (FL) addresses privacy and data-silo issues in the training of large language models (LLMs).

By Mingxiang Tao, Yu Tian, Wenxuan Tu, Yue Yang, Xue Yang, Xiangyan Tang

llms fine-tuning safety

Related stories