arXiv AI

Jailbreaking Open-Weight LLMs via Random Embedding Perturbations

arXiv AI
Jun 4

TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering

arXiv:2602. 06911v2 Announce Type: replace-cross Abstract: As increasingly capable open-weight large language models (LLMs) are deployed, improving their tamper resistance against unsafe modifications, whether accidental or intentional, becomes critical to minimize risks.

By Saad Hossain, Tom Tseng, Punya Syon Pandey, Samanvay Vajpayee, Matthew Kowal, Nayeema Nonta, Samuel Simko, Stephen Casper, Zhijing Jin, Kellin Pelrine, Sirisha Rambhatla
arXiv AI
Jul 9

NonTextual Target Attack

arXiv:2510. 02999v5 Announce Type: replace-cross Abstract: Existing gradient-based jailbreak attacks on Large Language Models (LLMs) typically optimize adversarial suffixes to align the LLM output with predefined target responses.

By Xinzhe Huang, Wenjing Hu, Tianhang Zheng, Kedong Xiu, Hongsheng Hu, Xiaojun Jia, Di Wang, Zhan Qin, Kui Ren
arXiv AI
Jul 23

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models

arXiv:2607. 19424v1 Announce Type: cross Abstract: The assessment of jailbreak attacks against large language models currently suffers from inconsistent evaluation criteria and methods, leading to unreliable estimates of attack success rates.

By Qingjia Huang, Jingyu Zhang, Jianguo Wu, Yakai Li, Weijuan Zhang, Yankai Rong, Junyi Yao, Shengzhi Zhang, Xiaoqi Jia
arXiv AI
Aug 19

Fair ASR: Re-Evaluating Black-Box Jailbreaks under Shared Target-Call Budgets

The paper introduces Fair-ASR, a protocol for evaluating black‑box jailbreak attacks using a shared target‑call budget, addressing the bias of prior studies that rely solely on attack success rate. Re‑evaluating 11 attacks under this protocol shows significant shifts in rankings and highlights that many methods are not efficient in both target and attacker calls. The authors then present ReCode, a budget‑efficient attack that combines desensitization rewriting with low‑cost primitives, achieving 85% ASR on GPT‑5 with only 7.19 attacker calls per request under a 20‑target‑call budget.

By Zhida He, Xiaoyu Wen, Han Qi, Ziyuan Zhou, Peng Yu, Jiajia Li, Chaochao Lu, Qiaosheng Zhang