arXiv AI
Jul 1
Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens
arXiv:2606. 30755v1 Announce Type: cross Abstract: Claw-like AI agents (e.
The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems
arXiv:2606. 26057v1 Announce Type: cross Abstract: AI agents are granted access to tools, APIs, and other infrastructure, making them active principals in those systems.
Related stories
arXiv AI
Jun 6
CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
arXiv:2601. 09923v3 Announce Type: replace Abstract: AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior.
arXiv AI
Sep 23
When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems
The paper discusses how large language model agents now act as privileged principals with kernel‑grade authority, yet lack the trusted mediation traditionally required for operating‑system security. It introduces a taxonomy that distinguishes between provenance‑based deterministic checks and content‑semantic checks, identifying a central mediation gap in distinguishing data from instruction and authorized from unauthorized actions. The authors argue that this gap creates an irreducible risk of undetected attacks whenever inputs and actions are not pre‑enumerated, and they propose defenses across runtime monitoring, architectural separation, and authorization while critiquing current evaluation practices. They extend the analysis to AI‑native operating systems where the model itself serves as the arbitration core, outlining design constraints, challenges, and a research agenda.
arXiv AI
Jun 29
On the Inseparability of Instructions and Data in Shared-Embedding Sequence Models
arXiv:2606. 27567v1 Announce Type: cross Abstract: Prompt injection is the top security risk for LLM-integrated applications, yet every defense proposed so far has been broken.
arXiv AI
Jul 13
The LLMbda Calculus: AI Agents, Conversations, and Information Flow
arXiv:2602. 20064v2 Announce Type: replace-cross Abstract: Large language models are increasingly deployed as agents: they plan, call tools, read untrusted data, and act on the results.
arXiv AI
Aug 13
Agent Safety Should Be a Runtime Contract
arXiv:2608. 11274v1 Announce Type: cross Abstract: The dominant paradigm treats AI safety as a property to be instilled during model training via RLHF, DPO, or Constitutional AI.
arXiv AI
Jun 12
Beyond Runtime Enforcement: Shield Synthesis as Defensibility Analysis for Adversarial Networks
arXiv:2606. 13621v1 Announce Type: new Abstract: Shielded reinforcement learning is typically presented as a runtime safety mechanism that compiles temporal-logic specifications into automata restricting an agent's actions.
arXiv AI
Jul 7
Governed MCP: Kernel-Level Tool Governance for AI Agents via Logit-Based Safety Primitives
arXiv:2604. 16870v2 Announce Type: replace-cross Abstract: AI agents increasingly call external tools (file system, network, APIs) through the Model Context Protocol (MCP).
arXiv AI
Jul 7
The Unverifiability of Artificial General Intelligence (AGI) Alignment, Static and Dynamic: From Trakhtenbrot's Wall to the Safety-Generality Tension
arXiv:2606. 28639v2 Announce Type: replace-cross Abstract: We establish the mathematical limits of AGI safety in two forms: verifying a fixed system, and verifying that a certified safety property persists once the system self-modifies.
arXiv AI
Sep 2
Delegation Without Trust: An Empirical Gap Analysis of Identity, Authorization, and Runtime Governance in Multi-Agent LLM Systems
The paper examines the security challenges of delegating authority to autonomous LLM agents that act on users’ behalf. It introduces a threat model with four adversaries and eight security requirements, demonstrates that current frameworks (LangGraph, CrewAI, AutoGen, MCP) fail to meet these standards, and presents an authorization broker that blocks all identified threats with minimal overhead. The broker is shown to resist numerous attacks and limits compromised sub‑agents to their delegated tasks, and its principles are implemented in VotalAI’s LLM Shield.
arXiv AI
Aug 10
NiyamAI - An Intent-Bound AI Agent with Cryptographically Verifiable Guardrails using Zero-Knowledge Proofs
arXiv:2608. 07167v1 Announce Type: new Abstract: Giving an AI agent the ability to send emails, query databases, or execute commands is useful--until the agent is tricked into doing something it shouldn't.
Hugging Face Trending Papers
Jun 11
Beyond Runtime Enforcement: Shield Synthesis as Defensibility Analysis for Adversarial Networks
Shielded reinforcement learning is typically presented as a runtime safety mechanism that compiles temporal-logic specifications into automata restricting an agent's actions. We argue this is the wrong product.