arXiv AI

Local Is Not a Sufficient Privacy Boundary: Governing OS-Integrated On-Device AI

arXiv:2606. 10173v1 Announce Type: cross Abstract: As AI systems move into operating systems, privacy no longer turns only on whether a model runs locally.

arXiv AI
Jul 28

Intent-Governed Tool Authorization for AI Agents

arXiv:2606. 22916v2 Announce Type: replace Abstract: AI agents increasingly act through external tools: they read private data, construct structured payloads, submit write requests, export records, and coordinate workflows across application boundaries.

By Genliang Zhu, Chu Wang
Hugging Face Trending Papers
Jun 22

Intent-Governed Tool Authorization for AI Agents

AI agents increasingly act through external tools: they read private data, construct structured payloads, submit write requests, export records, and coordinate workflows across application boundaries. Existing authorization mechanisms usually ask whether an integration credential, app, or token can call a tool.

arXiv Machine Learning
2d ago

Privacy in Personalized AI Is a System Property, Not Just a Model Property

The paper argues that privacy in personalized AI should be viewed as a system-level issue rather than just a model-level one. It identifies four interconnected privacy‑risk channels in personalized AI and proposes four system‑level requirements—interaction trajectories, internal information flows, indirect leakage, and the privacy‑utility trade‑off—for evaluating privacy. The authors call for these requirements to be systematically incorporated into privacy audits of personalized AI systems.

By Guillaume Salha-Galvan, Jiaying Xu
arXiv AI
Sep 23

When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems

The paper discusses how large language model agents now act as privileged principals with kernel‑grade authority, yet lack the trusted mediation traditionally required for operating‑system security. It introduces a taxonomy that distinguishes between provenance‑based deterministic checks and content‑semantic checks, identifying a central mediation gap in distinguishing data from instruction and authorized from unauthorized actions. The authors argue that this gap creates an irreducible risk of undetected attacks whenever inputs and actions are not pre‑enumerated, and they propose defenses across runtime monitoring, architectural separation, and authorization while critiquing current evaluation practices. They extend the analysis to AI‑native operating systems where the model itself serves as the arbitration core, outlining design constraints, challenges, and a research agenda.

By Li Zhang, Yang Sun, Jie Shi
arXiv AI
Sep 25

AgentKernel: The Trust-Native Agentic Operating System

AgentKernel proposes a trust‑native operating system for AI agents, arguing that current governance layers are insufficient because they share the same process trust boundary as the agents. The OS introduces a mandatory enforcement boundary organized into four pillars—Identity, Perception, Cognition, and Execution—each adapting classical OS security principles to address semantic‑level failures such as prompt injection, memory poisoning, and tool misuse. By wrapping the agent lifecycle in this structured, non‑bypassable framework, AgentKernel aims to provide a unified security layer that can enforce identity, input mediation, memory governance, and execution control across the entire agent lifecycle.

By Zhenhua Zou, Sheng Guo, Qiuyang Zhan, Lepeng Zhao, Shuo Li, Zhuotao Liu
arXiv Machine Learning
Aug 31

Not to Break, but to Attest: Adversarial Probes for Privacy-Preserving LLM Verification

The paper introduces a privacy‑preserving zk‑SNARK audit framework that uses adversarial‑style probes to detect logit drift between an approved large language model and a modified deployment. It offers three probe families—token‑based (black‑box), embedding‑based (gray‑box), and stress probes (partial white‑box)—allowing users to balance sensitivity, access, and cost. Experiments across LLM architectures and GPU platforms show token‑based probes achieve the highest mean sensitivity while remaining practical in a black‑box setting, with Groth16 proving times scaling modestly from 1.02 to 1.78 seconds and constant proof size.

By Cameron Wilding, Mina Shaker, Fatemeh Ganji