arXiv AI

Arbitrary Cipher Attacks Against Large Language Models Do Not Require Fine-Tuning

The paper reports that large language models can acquire cipher-based covert communication skills without fine‑tuning, using prompting or in‑context learning instead. This enables new jailbreak attacks that bypass alignment safeguards by encrypting harmful requests, making them appear as nonsensical text to harmfulness classifiers. The authors demonstrate successful attacks against frontier models from Anthropic, Google, and OpenAI.

arXiv AI

Sep 21

HE-Guardrail: A Homomorphic Guardrail Against Jailbreak Attacks for Encrypted Large Language Model Inference

HE-Guardrail is a framework that applies homomorphic encryption to enforce guardrails against jailbreak attacks during encrypted large language model inference. It evaluates guardrail mechanisms—Llama Guard, JBShield, and GradSafe—directly on encrypted data, deciding whether to return the model’s response to the client. The approach preserves confidentiality while closely matching the decisions of plaintext guardrails, offering varied security-efficiency-utility trade‑offs.

By Byeongseo Min, Yongwoo Lee, Young-Sik Kim, Yongjune Kim

llms safety

arXiv Machine Learning

Jun 4

Bypassing Prompt Guards in Production with Controlled-Release Prompting

arXiv:2510. 01529v3 Announce Type: replace Abstract: Ball et al.

By Jaiden Fairoze, Sanjam Garg, Keewoo Lee, Mingyuan Wang

llms safety

arXiv AI

Jul 7

Context Misleads LLMs: The Role of Context Filtering in Maintaining Safe Alignment of LLMs

arXiv:2508. 10031v2 Announce Type: replace-cross Abstract: While Large Language Models (LLMs) have shown significant advancements in performance, various jailbreak attacks have posed growing safety and ethical risks.

By Jinhwa Kim, Ian G. Harris

llms fine-tuning benchmarks safety

arXiv AI

Sep 10

SAFEGuard: Detect Optimization-Based Jailbreak Attacks Through Harmful Semantic Analysis and Fluency Measurement

arXiv:2609.05850v1 Announce Type: cross Abstract: Despite the significant efforts devoted to aligning large language models (LLMs) with human values and ensuring safe deployment, recent work has reve...

By Quoc Viet Vo, Trung Le, Damith C. Ranasinghe, Ehsan Abbasnejad

llms benchmarks safety

arXiv Computation and Language

Sep 23

Dynamic Deep Prompt Optimization for Defending Against Jailbreak Attacks on LLMs

arXiv:2609.26185v1 Announce Type: cross Abstract: Large Language Models (LLMs) demonstrate impressive capabilities across many applications but remain vulnerable to jailbreak attacks, which elicit ha...

By Doniyorkhon Obidov, Honggang Yu, Xiaolong Guo, Kaichen Yang

llms rag fine-tuning safety

arXiv AI

Jun 11

Grammar-Constrained Decoding Can Jailbreak LLMs into Generating Malicious Code

arXiv:2606. 11817v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for code generation, raising concerns that they may be misused to produce malicious code.

By Yitong Zhang, Shiteng Lu, Jia Li

llms benchmarks safety

arXiv Computation and Language

Sep 22

BreakFun: Jailbreaking LLMs via Object Instantiation under Simulated Code Execution

arXiv:2510.17904v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are widely used because they process structures, syntax and code well, but this same ability also makes them par...

By Amirkia Rafiei Oskooei, Mehmet S. Aktas

llms benchmarks safety

arXiv AI

Aug 11

Stealing Reasoning Traces from Proprietary LLM APIs

arXiv:2608. 09867v1 Announce Type: cross Abstract: Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage.

By Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner, Joachim Schaeffer, Ameya Prabhu, Jonas Geiping, Maksym Andriushchenko

llms rag agents efficiency safety

arXiv AI

Jun 3

Patcher: Post-Hoc Patching of Backdoored Large Language Models

arXiv:2606. 02995v1 Announce Type: cross Abstract: Large language models remain vulnerable to jailbreak backdoor attacks, where adversaries poison safety alignment data to embed hidden triggers that bypass safety mechanisms.

By Anjun Gao, Yueyang Quan, Yufei Xia, Zhuqing Liu, Minghong Fang

llms fine-tuning safety

arXiv AI

Aug 20

Jailbreaking in the Haystack

The paper "Jailbreaking in the Haystack" introduces NINJA, a jailbreak technique that exploits long-context language models by appending benign, model-generated content to harmful user goals. It demonstrates that the position of harmful goals within the context is crucial for safety, and shows that NINJA significantly boosts attack success rates on models such as LLaMA, Qwen, Mistral, and Gemini. Unlike previous methods, NINJA is low-resource, transferable, less detectable, and compute‑optimal, revealing that carefully crafted benign long contexts can expose fundamental vulnerabilities in modern LMs.

By Rishi Rajesh Shah, Chen Henry Wu, Shashwat Saxena, Ziqian Zhong, Alexander Robey, Aditi Raghunathan

llms agents benchmarks safety

arXiv AI

Jul 23

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models

arXiv:2607. 19424v1 Announce Type: cross Abstract: The assessment of jailbreak attacks against large language models currently suffers from inconsistent evaluation criteria and methods, leading to unreliable estimates of attack success rates.

By Qingjia Huang, Jingyu Zhang, Jianguo Wu, Yakai Li, Weijuan Zhang, Yankai Rong, Junyi Yao, Shengzhi Zhang, Xiaoqi Jia

llms benchmarks safety

arXiv Machine Learning

Jun 30

Sparse Autoencoders are Capable LLM Jailbreak Mitigators

arXiv:2602. 12418v2 Announce Type: replace-cross Abstract: Jailbreak attacks remain a persistent threat to large language model safety.

By Yannick Assogba, Jacopo Cortellazzi, Javier Abad, Pau Rodriguez, Xavier Suau, Arno Blaas

llms diffusion safety

Related stories