Hugging Face Blog

2024 Security Feature Highlights

arXiv AI
Aug 19

Future-Back Threat Modeling: A Foresight-Driven Security Framework

Future-Back Threat Modeling (FBTM) is a predictive security framework that starts with envisioned future threat states and works backward to uncover assumptions, gaps, blind spots, and vulnerabilities in current defense architectures. It aims to reveal both known unknowns and unknown unknowns, including emerging tactics, techniques, and procedures, thereby improving the predictability of adversary behavior under future uncertainty. By anticipating future threats such as AI, information warfare, and supply chain attacks, FBTM helps security leaders make informed decisions today to build more resilient security postures for the future.

By Vu Van Than
arXiv AI
Sep 11

CS-Guard: Benchmarking LLM Guardrails for Code Generation Security

CS-Guard is a new benchmark that systematically evaluates guardrails for code generation security, covering 1,000 malware-generation prompts, 7 jailbreak attacks, and a novel fictional scenario attack (FSA) for text-to-code generation, as well as 331 code prompts for code-to-code generation. The study empirically tests nine guardrails across seven large language models, finding that many guardrails fail to prevent malicious code generation, with attack success rates reaching about 50% for text-to-code and up to nearly 100% for code-to-code and FSA scenarios. CS-Guard introduces a modular three-layer guardrail taxonomy and releases its benchmark and data to support future research.

By Jinyang Li, Mingyu Guo, Hung X. Nguyen
OpenAI Blog
Sep 1

Path to Astra: critical capabilities and frontier safeguards

The OpenAI Blog article titled "Path to Astra: critical capabilities and frontier safeguards" announces that Astra is the first OpenAI model to meet the Critical cybersecurity capability threshold under the Preparedness Framework. It highlights that Astra incorporates stronger safeguards for its release, ensuring higher security standards. The piece underscores the model’s compliance with stringent cybersecurity criteria.

arXiv AI
Aug 5

AI Security Leaderboard: Methodology, Results and Minimal Standard

arXiv:2608. 03070v1 Announce Type: cross Abstract: Frontier AI model developers increasingly rely on layered safeguards to prevent catastrophic misuse, but little public evidence exists on how much protection these safeguards provide, or how consistently across developers.

By Jasper Timm, Lukas Struppek, Ziwei Xu, Grace Cheong, Oscar Mata, Dan Zhao, Mick Yang, Isadora De Andrade, Xiaojun Jia, Yiming Li, Samuel Bauer, Heather McIntyre, Adam Gleave, Edward Yee, Kellin Pelrine
arXiv AI
Jul 7

SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses

arXiv:2510. 15476v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used as interfaces to information, code, and real-world services, making prompt-level security failures a practical concern.

By Hanbin Hong, Shuang Wu, Shuya Feng, Nima Naderloui, Shenao Yan, Jingyu Zhang, Ali Arastehfard, Heqing Huang, Yuan Hong
arXiv AI
Nov 2

RiskyPy: Toward Model-Calibrated Prompt Generation for Joint Security and Functionality Evaluation of LLM-Generated Code

arXiv:2610.40735v1 Announce Type: cross Abstract: Evaluating approaches that improve the security of LLM code generation requires datasets that jointly assess security and functionality and remain ad...

By Zacharie Chenail-Larcher, Daniele Lunghi, Melissa Tessa, Quentin Sti\'evenart, Jacques Klein, Tegawend\'e F. Bissyand\'e, Florent Avellaneda, Naouel Moha
arXiv AI
Jul 9

Large Language Models (LLMs) and Generative AI in Cybersecurity and Privacy: A Survey of Dual-Use Risks, AI-Generated Malware, Explainability, and Defensive Strategies

arXiv:2607. 06963v1 Announce Type: cross Abstract: Large Language Models (LLMs) and generative AI (GenAI) systems, such as ChatGPT, Claude, Gemini, LLaMA, Copilot, Stable Diffusion by OpenAI, Anthropic, Google, Meta, Microsoft, Stability AI, respectively, are revolutionizing cybersecurity, enabling both automated defense and sophisticated attacks.

By Kiarash Ahi, Saeed Valizadeh