arXiv AI

Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls

arXiv:2607. 24343v1 Announce Type: cross Abstract: Language-model agents act through structured tool calls whose arguments carry different risks.

arXiv AI
Aug 24

ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents

ClawSentry is an open‑source, framework‑agnostic security supervision gateway designed to protect autonomous large language model (LLM) agents from progressive risks that can arise at four points in the agent control loop: skill admission, invocation‑time intent, execution‑time effect, and post‑action consequence. It introduces a multi‑tier decision engine—deterministic L1, rule‑anchored L2, and read‑only L3—alongside a First‑Use Skill Package Review (FSPR) and an Agent Harness Protocol (AHP) that applies a single policy across multiple agent runtimes without modifying their internals. Evaluation on SkillInject and the SkillsSafety benchmark shows that ClawSentry significantly reduces contextual adversarial skill risk (ASR) while maintaining high task success rates (TSR).

By Kai Wang, Zeming Wei, BiaoJie Zeng, Chang Jin, An Wang, Xiaokun Luan, Zhixiao Lin, Jingjing Qu, Xia Hu, Xingcheng Xu
arXiv AI
Sep 25

Progressive Skill Discovery as Access Control for Tool-Using LLM Agents: Structural Governance through Role-Scoped Capability Delivery

The paper introduces skilder, a framework that organizes LLM agent capabilities into role‑scoped bundles of skills, tools, and instructions, with explicit limits. Agents start with a minimal role catalog, discover the roles needed for a task, and receive the associated tools only through a single MCP server, ensuring deterministic enforcement of scope. Experiments on 13 tasks with six models show that skilder’s authorization layer prevents unauthorized tool calls and parameter violations while maintaining flexibility through dynamic cross‑role capability acquisition.

By Michael Stettler, Benjamin Girardet, Jonas Canton, Nicolas Corod
arXiv AI
Jun 18

SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents

arXiv:2606. 18356v1 Announce Type: cross Abstract: Tool-using language-model agents introduce security failures that go beyond unsafe text: they can disclose protected objects, write persistent memory, send messages, modify databases, or trigger harmful code and tool effects.

By Yuchuan Tian, Mengyu Zheng, Haocheng Mei, Ye Yuan, Chao Xu, Xinghao Chen, Hanting Chen, Yu Wang
arXiv AI
Aug 19

HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety

HarnessRisk is a lifecycle-oriented benchmark for evaluating safety in agent harnesses that manage tools, extensions, state, permissions, and external actions. It defines six operational phases—Harness Configuration, Capability Extension, Runtime Operation, State Persistence, Action Control, and Incident Recovery—and includes 128 sandboxed cases pairing benign user objectives with adversarial instructions. Across three harnesses, six language models, and 14 configurations, attack success rates vary from 12.6% to 80.9%, with the most vulnerable phase being Harness Configuration. "whyItMatters":"The benchmark demonstrates that safety failures can arise in multiple harness responsibilities and that even explicit risk detection does not guarantee safe action, underscoring the need for comprehensive evaluation across model and harness configurations."

By Yajing Bai, Jinhao Duan, Jie Peng, Xianfeng Wu, Sijia Liu, Song Wang, Tianlong Chen
arXiv AI
Aug 19

TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generation

TRUSS is a framework that generates and verifies automated agent skills, ensuring they are both functionally effective and safe. It evaluates candidate skills against source evidence and nine safety properties, then tests them in a controlled environment to capture execution traces and identify failures. The system iteratively refines skills based on these results, achieving high precision in vulnerability detection and significantly improving task performance and security rates.

By Zhibo Zhang, Zhen Ouyang, Ling Shi, Kailong Wang