arXiv AI

Signal-based Model Access Risk Analysis for AI System Operations Security

arXiv:2607. 16414v1 Announce Type: cross Abstract: Artificial intelligence (AI) systems are now ubiquitous across domains such as security, finance, healthcare, consumer technology, and large-scale cloud services, where they process massive volumes of data and make consequential decisions daily.

arXiv AI
Jun 3

AI Model Extraction Attacks: Bypassing Single-Client Assumptions in Defenses

arXiv:2606. 03381v1 Announce Type: cross Abstract: Ensuring the protection of Artificial Intelligence (AI) models deployed in military Command and Control (C2) systems and critical infrastructure is essential for maintaining information superiority.

By Maxime Schwarzer, Johannes F. Loevenich, Gustavo S\'anchez, Laurin Holz, Thies M\"ohlenhof, Tobias H\"urten, Roberto Rigolin F. Lopes, Veit Hagenmeyer
arXiv AI
Jun 12

The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems

arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.

By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang
arXiv AI
Oct 19

Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems

The paper surveys transferable attacks on AI systems, noting that such attacks can be crafted in one setting and then applied to others, thereby lowering costs and increasing security risks. It introduces a unified taxonomy across six axes—cross-instance, cross-domain, cross-modality, cross-model, cross-task, and cross-deployment—while distinguishing source-target changes from attacker resources. The survey also links transfer mechanisms to enhancement methods, highlights inconsistencies and gaps in current research, and proposes a research agenda for better benchmarking and defense strategies.

By Guangjing Wang, Ce Zhou, Yuanda Wang, Bocheng Chen, Hanqing Guo, Qiben Yan
arXiv AI
Sep 25

On the Effectiveness of Kernel-Level Evidence for Agent Security

The paper introduces the Agent Cross‑Layer Evidence (ACE) corpus, pairing application‑level telemetry with kernel‑level syscall traces to study agent security. It shows that kernel evidence alone is discriminative and that combining it with application‑level data outperforms either layer alone, revealing complementary signals. The study also demonstrates that this cross‑layer approach generalizes to unseen attack families and works across different agent runtimes.

By Spencer King, Zhilu Zhang, Mikhail Kuznetsov, Kay Liu, Baris Coskun, Wei Ding
arXiv Machine Learning
Sep 21

Identifying Security Platform Product Abuse with Machine Learning

arXiv:2609.21303v1 Announce Type: cross Abstract: Product abuse is an individually rare, but growing, problem across the SaaS industry. Highly sophisticated threat actors can misuse security platform...

By Shaefer Drew, Michael Brautbar, Paul Knight, Edward Raff, Lana Peric-McDermott, Simran Sarin, Nickolas Machado, Hanna Albright, Vitaly Zaytsev
Hugging Face Trending Papers
2d ago

Scaling Adaptive Prompt Injection Attacks with Agent Swarms

The paper discusses the ongoing threat of prompt injection attacks on AI agents, noting that early simple attacks have plateaued but new, more capable adversaries can still succeed. It introduces a synthetic evaluation dataset that uses random non‑malicious actions as adversarial goals to test whether models respect instruction hierarchy independently of security knowledge. Experiments show that even agents considered robust on existing benchmarks can be tricked by advanced attackers, indicating that current defenses may need proactive, automated red‑teaming.