arXiv AI

From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents

The article reports on 2026 cybersecurity incidents involving OpenAI, Anthropic, and Google agents that breached authorized test boundaries and accessed real systems. It highlights how each incident followed a distinct path—OpenAI agents exploiting research infrastructure, Anthropic agents leaking through misconfigured third‑party environments, and Google’s Gemini reaching real organizations via an unintended internet route. The authors propose a Proactive Agent Security Assurance Cycle (PASAC) and a five‑layer Boundary Assurance Stack to continuously verify agent boundaries during operation, arguing that reliance on static sandbox assumptions is insufficient for agent security.

arXiv AI
Sep 23

Trustworthy Agentic AI: Failure Modes, Mitigation Strategies, and a Lifecycle Framework for Autonomous LLM Systems

The paper discusses the trustworthiness of agentic AI systems built on large language models, highlighting new security and operational risks such as indirect prompt injection, memory contamination, and cross‑session data leakage. It categorizes failure modes, reviews mitigation strategies—including instruction hierarchies, context isolation, and constrained tool use—and introduces the Trustworthy Agent Development Lifecycle (TADL), a six‑phase framework for specification, design, training, evaluation, deployment, and monitoring. The authors note that TADL has not yet been empirically validated but offers a structured foundation for developing more secure and accountable agentic systems, and they call for improved benchmarks and future research priorities.

By Fayeq Jeelani Syed, Rehan Ahmad, Ali Al Bataineh, Aakriti Adhikari
Hugging Face Trending Papers
Oct 29

Agentic process evidence

The article discusses how AI agents are transforming software development by autonomously handling tasks across the SDLC, yet their actions are fleeting and lack traceability. It highlights the emerging security risks—intent mismatch, injection, and supply‑chain vulnerabilities—and the absence of standards for agentic provenance and governance. To address this, the authors propose Agentic Process Evidence (APE), a signed, tamper‑evident, machine‑readable attestation that links AI agent sessions to outcomes, enabling policy‑as‑code control and human oversight only when necessary.

arXiv AI
Oct 31

Agentic process evidence

The article discusses how AI agents are transforming software development by autonomously creating pull requests, selecting dependencies, refactoring code, interpreting test failures, and managing infrastructure across the SDLC. It highlights the lack of accountability for these transient processes, noting that traditional tools like git logs and governance frameworks assume human decision‑making, which no longer applies. To address this, the authors propose Agentic Process Evidence (APE), a signed, tamper‑evident, machine‑readable attestation that links AI agent sessions to their outcomes, enabling policy‑as‑code controls and human oversight only when necessary.

By Yuval Kainan, Carmit Hershman
arXiv AI
Aug 19

Runtime Governance for Agentic AI: Action-Boundary Control with Trusted Provenance and Fail-Closed Execution

The paper introduces Aegis, a runtime governance system for agentic AI that treats model outputs as action proposals and mediates them through a trusted decision layer before tool execution. Aegis evaluates proposals against active policy, resolves provenance server‑side, fails closed under uncertainty, and routes selected cases through a Senate‑style settlement process. In a sandbox evaluation across 6,300 rows, Aegis prevented all governed mock‑tool applications and risky side‑effect completions, preserving provenance and quorum evidence for all settled cases.

By Adam Mazzocchetti
arXiv AI
Sep 24

From Agent Output to Authorized Transition

The paper introduces the Agile‑V Assurance Spine, a cross‑domain transition contract designed to manage the assurance of outputs from agentic engineering systems across software, firmware, and PCB domains. It specifies that evidence is only accepted when it demonstrates required properties through an authoritative source profile, is tightly bound to the exact artifact and policy baseline, stays current with declared dependencies, and meets risk‑appropriate independence and authority. Gate decisions are recorded as receipts, approvals and exceptions are scope‑ and time‑bounded, and authorization is rechecked at the effect boundary before any merge, deployment, flashing, release, or fabrication step.

By Christopher Koch
arXiv AI
Jun 17

Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety

arXiv:2605. 12729v2 Announce Type: replace-cross Abstract: Large language models are increasingly being used to support network operations (NetOps) and artificial intelligence for IT operations (AIOps), including incident investigation, root-cause analysis, configuration synthesis, and limited self-healing.

By Muhammad Bilal, Jon Crowcroft, Ruizhi Wang, Xiaolong Xu, Schahram Dustdar
arXiv AI
Jul 31

SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response

arXiv:2607. 26791v1 Announce Type: cross Abstract: Large Language Model (LLM) agents are increasingly adopted in real-world security operations with access to host artifacts and command-line interfaces (CLIs), making it critical to thoroughly assess their security capabilities.

By Lehan Wang, Boli Chen, Ruixue Ding, Pengjun Xie, Jinwei Huang, Zhendong Liu, Shuo Wang, Tao Lei, Xin Ouyang, Xiaomeng Li