arXiv AI

When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems

The paper discusses how large language model agents now act as privileged principals with kernel‑grade authority, yet lack the trusted mediation traditionally required for operating‑system security. It introduces a taxonomy that distinguishes between provenance‑based deterministic checks and content‑semantic checks, identifying a central mediation gap in distinguishing data from instruction and authorized from unauthorized actions. The authors argue that this gap creates an irreducible risk of undetected attacks whenever inputs and actions are not pre‑enumerated, and they propose defenses across runtime monitoring, architectural separation, and authorization while critiquing current evaluation practices. They extend the analysis to AI‑native operating systems where the model itself serves as the arbitration core, outlining design constraints, challenges, and a research agenda.

arXiv AI
Sep 25

AgentKernel: The Trust-Native Agentic Operating System

AgentKernel proposes a trust‑native operating system for AI agents, arguing that current governance layers are insufficient because they share the same process trust boundary as the agents. The OS introduces a mandatory enforcement boundary organized into four pillars—Identity, Perception, Cognition, and Execution—each adapting classical OS security principles to address semantic‑level failures such as prompt injection, memory poisoning, and tool misuse. By wrapping the agent lifecycle in this structured, non‑bypassable framework, AgentKernel aims to provide a unified security layer that can enforce identity, input mediation, memory governance, and execution control across the entire agent lifecycle.

By Zhenhua Zou, Sheng Guo, Qiuyang Zhan, Lepeng Zhao, Shuo Li, Zhuotao Liu
arXiv AI
Aug 19

Runtime Governance for Agentic AI: Action-Boundary Control with Trusted Provenance and Fail-Closed Execution

The paper introduces Aegis, a runtime governance system for agentic AI that treats model outputs as action proposals and mediates them through a trusted decision layer before tool execution. Aegis evaluates proposals against active policy, resolves provenance server‑side, fails closed under uncertainty, and routes selected cases through a Senate‑style settlement process. In a sandbox evaluation across 6,300 rows, Aegis prevented all governed mock‑tool applications and risky side‑effect completions, preserving provenance and quorum evidence for all settled cases.

By Adam Mazzocchetti
arXiv AI
Sep 2

Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systems

The paper examines the security challenges of delegating authority to autonomous LLM agents that act on users’ behalf. It introduces a threat model with four adversaries and eight security requirements, demonstrates that current frameworks (LangGraph, CrewAI, AutoGen, MCP) fail to meet these standards, and presents an authorization broker that blocks all identified threats with minimal overhead. The broker is shown to resist numerous attacks and limits compromised sub‑agents to their delegated tasks, and its principles are implemented in VotalAI’s LLM Shield.

By Panduranga Sai Varma Dantuluri, Jyotirmoy Sundi
arXiv AI
Oct 5

Beyond Predefined Sinks: Security-Aware Dependency Analysis for LLM Agents

The paper introduces AgentSecGraph, a static analysis framework that builds a Security-Aware Agent Dependency Graph (Security-ADG) for each security-sensitive operation in large language model (LLM) agents. It enriches operation identity with agent relevance, source and dependency evidence, trust-boundary context, guard evidence, and external-effect semantics. The authors also present AgentSecBench, a corpus of 67 real-world LLM-agent repositories, and demonstrate that their analyzer identifies thousands of operation candidates, recovers substantial dependency and guard evidence, and distinguishes guarded behaviors from vulnerabilities with high accuracy.

By Hang Cui
arXiv AI
Oct 2

PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents

The paper introduces PACE, a Provenance-Aware Capability Enforcement system designed to secure tool-using large language model agents by mediating every tool call before execution. PACE employs path confinement to limit influence paths and verifies effects against authenticated authority, distinguishing certified execution contracts from evaluated configurations. Experiments on eight agent‑security benchmarks show that the evaluated configuration reduces attack success in most cases while maintaining near‑native utility.

By Fengpeng Li, Qizhou Wang, Yuke Hu, Kemou Li, Jun Liu, Haiwei Wu, Jiantao Zhou, Di Wang
arXiv AI
Sep 23

ActGov: Governing LLM Agent Actions via Policy-Constrained Validation

ActGov is a runtime enforcement framework that validates each action proposed by a large language model (LLM) agent before it interacts with external tools, ensuring that actions stay within task‑scoped authorization boundaries and comply with dynamically constructed policies. It builds policies from tool specifications, benign tasks, and failure traces, verifying updates via SMT‑based counterexample checking. In evaluations on AgentDojo and AgentDyn benchmarks, ActGov consistently reduces indirect prompt‑injection attack success while maintaining task utility, outperforming existing defenses.

By Kaiyuan Zhang, Yuke Peng, Ke Jiang, Yinqian Zhang