arXiv AI

Self-State Attacks on Self-Hosted AI Agents: How Far Can OS Defenses Go?

arXiv:2607. 17986v1 Announce Type: cross Abstract: Self-hosted AI agents read and write their own memory and configuration files to function.

arXiv AI
Sep 17

Reflections on Trusting Trust, Revisited: Contaminating Self-Modifying AI Coding Agents with Poisoned Benchmarks

The paper revisits Thompson’s classic compiler back‑door attack in the context of self‑modifying AI coding agents. By poisoning the benchmarks used for self‑evaluation, the authors demonstrate that agents such as the Darwin Gödel Machine, Self‑Improving Coding Agent, and Hyperagents can be coaxed into generating vulnerable code, even on clean, held‑out tasks. Experiments show that the contamination can persist after subsequent clean training, highlighting the need for more robust agent designs.

By Franziska Roesner, Tadayoshi Kohno
arXiv AI
Sep 24

Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory

The paper introduces ShadowMem, a defensive framework that protects large language model agents from long-horizon threats by maintaining a dedicated safety-focused memory. Inspired by the shadow stack concept, ShadowMem stores safety-critical context throughout an agent’s execution and uses this shadow memory to evaluate the risk of upcoming actions before they are carried out. Experiments show that ShadowMem outperforms existing defenses in detection accuracy, detects most attacks early, and adds minimal overhead to agent performance.

By Yuhui Wang, Tanqiu Jiang, Jiacheng Liang, Charles Fleming, Ting Wang
arXiv AI
Oct 5

Pincer: Resource Authorization for Agents using a Digital Twin

Pincer is a defense mechanism for coding agents that operates at the resource layer, complementing existing tool‑call defenses. It uses a digital twin—an isolated‑context model that learns and enforces dynamic, user‑specific least‑privilege policies—to act as a proxy for the agent’s permission requests. The authors introduce a user‑centric dataset to emulate the learning phase and demonstrate that Pincer outperforms several baselines in both security and utility, especially against adversarial attack types.

By Mayank Rathee, Alexander Stepanov, Shalin Madabhavi, Jinhao Zhu, Raluca Ada Popa, Ion Stoica