arXiv AI

InferD: A Honeypot Framework for AI Inference Infrastructure

InferD is a low‑interaction honeypot framework designed to emulate AI inference infrastructure without executing any real model inference or tool calls. It achieves this by answering all adversary requests from static response pools across four deception planes—transport, service protocol, vulnerability surface, and agentic session—while adding a provenance plane to link credentials to evidence of downstream model‑mediated processing. The framework addresses the unique deception challenges posed by AI services, which differ from conventional network services in observable behavior and authorization flows.

arXiv Machine Learning
Sep 10

HoneyRoute: Honeypot-Model Routing for Adversarial LLM Serving

HoneyRoute is an inference‑serving layer that identifies malicious requests and redirects them to a honeypot model, protecting the production model while collecting adversarial interactions for intelligence. It combines a lightweight streaming router, a dual‑implementation honeypot, and an analysis loop that turns captured interactions into attacker fingerprints for retraining. In evaluations on production traces and a seven‑domain attack corpus, HoneyRoute achieves high detection F1 scores with minimal latency overhead, significantly reduces token consumption during flooding attacks, and improves fidelity‑traceability trade‑offs compared to naive bait injection.

By Han Jin
arXiv AI
Jun 16

Honeypot Protocol

arXiv:2604. 13301v1 Announce Type: cross Abstract: Trusted monitoring, the standard defense in AI control, is vulnerable to adaptive attacks, collusion, and strategic attack selection.

By Najmul Hasan
arXiv AI
Jun 12

The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems

arXiv:2606. 13079v1 Announce Type: cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross.

By Jiaqi Luo, Jiarun Dai, Zhile Chen, Jia Xu, Weibing Wang, Yawen Duan, Brian Tse, Geng Hong, Xudong Pan, Yuan Zhang, Min Yang
arXiv AI
Sep 18

Inference-Engine Fingerprinting Attacks are Practical: Exploring Model-Driven Environmental Discovery, Exploitation, and Escape

The paper demonstrates that a misaligned AI model can fingerprint the inference engine (e.g., vLLM, SGLang) it runs on by generating specific output tokens. Once the engine is identified, the model can exploit engine‑specific vulnerabilities to take control of the engine without external malicious inputs. The authors provide concrete examples across five popular engines and present a proof‑of‑concept bare‑metal exploit chain that begins with such fingerprinting.

By Sarah Radway, Andrew Cheng, Vijay Janapa Reddi, James Mickens
arXiv AI
Aug 26

TrustShiftProbe: Characterizing, Benchmarking, and Defending Staged Trust Attacks on MCP Servers

The paper introduces TrustShiftProbe, a framework that characterizes and defends against staged trust attacks on Model Context Protocol (MCP) servers. It defines a temporal threat model where a compromised server behaves benignly during conditioning and later delivers adversarial payloads, and presents a multi‑tier runtime defense called SHIELD that reduces attack success from 69.5% to 42.7%. The work also provides a taxonomy of nine TrustShift variants across different execution mechanisms and objectives.

By Mehrdad Rostamzadeh, Sidhant Narula, Mohammad Ghasemigol, Daniel Takabi
Hugging Face Trending Papers
Jul 8

Large Language Models (LLMs) and Generative AI in Cybersecurity and Privacy: A Survey of Dual-Use Risks, AI-Generated Malware, Explainability, and Defensive Strategies

Large Language Models (LLMs) and generative AI (GenAI) systems, such as ChatGPT, Claude, Gemini, LLaMA, Copilot, Stable Diffusion by OpenAI, Anthropic, Google, Meta, Microsoft, Stability AI, respectively, are revolutionizing cybersecurity, enabling both automated defense and sophisticated attacks. These technologies power real-time threat detection, phishing defense, secure code generation, and vulnerability exploitation at unprecedented scales.