arXiv AI
ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 27
V$^3$AST-Bench: Verifiable, Valid, and Versatile Safety Evaluation for Coding Agents
arXiv:2610.35188v1 Announce Type: cross Abstract: Large language models increasingly act through code, modifying repositories, executing programs, and interacting with services in agent form. Their a...
arXiv Machine Learning
Jun 19
AgentArmor: A Framework, Evaluation, \& Mitigation of Coding Agent Failures
arXiv:2606. 19380v1 Announce Type: cross Abstract: Software engineering and deployment are increasingly being delegated to AI coding agents.
Hugging Face Trending Papers
Jul 22
OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills
LLM-based agents leverage third-party skills to extend their capabilities in open-world scenarios. However, third-party skills can introduce extra security vulnerabilities, as seemingly harmless skills can contain latent safety risks that only emerge during actual execution.
arXiv AI
Aug 24
ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents
ClawSentry is an open‑source, framework‑agnostic security supervision gateway designed to protect autonomous large language model (LLM) agents from progressive risks that can arise at four points in the agent control loop: skill admission, invocation‑time intent, execution‑time effect, and post‑action consequence. It introduces a multi‑tier decision engine—deterministic L1, rule‑anchored L2, and read‑only L3—alongside a First‑Use Skill Package Review (FSPR) and an Agent Harness Protocol (AHP) that applies a single policy across multiple agent runtimes without modifying their internals. Evaluation on SkillInject and the SkillsSafety benchmark shows that ClawSentry significantly reduces contextual adversarial skill risk (ASR) while maintaining high task success rates (TSR).
Hugging Face Trending Papers
Aug 31
Beyond the Payload: How User Invocation Shapes Coding Agent Vulnerability to Repository Poisoning
Coding agents are increasingly used for software engineering tasks, including bootstrapping projects from third-party repositories whose integrity cannot be assumed. Prior work on repository poisoning...
Hugging Face Trending Papers
Aug 18
TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation
TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It first checks functional claims against evidence and evaluates artifacts against nine safety properties, then tests admitted skills in a controlled environment to capture execution traces and identify failures. The approach achieves perfect precision and recall in vulnerability detection, significantly reduces attack success rates, and boosts task effectiveness and security rates in skill generation benchmarks.