SafeEvolve is an experience-driven framework that co‑evolves a harness and policy to align large‑language‑model agents with safety goals. It uses completed on‑policy trajectories to update safety prompts and hierarchical skills, then applies a two‑stage SFT‑RL training loop that bootstraps the policy with the evolved harness and refines it through verifier‑augmented rewards. Experiments on agentic safety benchmarks show that SafeEvolve improves the safety‑utility tradeoff, achieving a three‑fold reduction in ASR on AgentDojo for Qwen3.5‑4B while increasing benign utility from 59.79% to 61.86%.
By Qinghua Mao, Wanying Qu, Dadi Guo, Leitao Yuan, Qingyu Liu, Yu Li, Guanxu Chen, Yanwei Fu, Xi Lin, Xia Hu, Dongrui Liu
arXiv:2609.38291v1 Announce Type: cross
Abstract: Large language model (LLM) agents are vulnerable to safety risks such as injected malicious instructions or misleading information, motivating runtim...
By Zhuo Liu, Moxin Li, Zhixin Ma, Wentao Shi, Wenjie Wang, Fuli Feng
SafeEvolve is an experience-driven framework that co‑evolves a harness and policy to improve safety alignment for LLM‑based agents. It uses on‑policy trajectory safety evidence to update safety prompts and hierarchical skills, producing auditable harness artifacts. The policy is trained via a two‑stage SFT‑RL pipeline that bootstraps with the evolved harness and then refines behavior through verifier‑decomposed rewards, yielding a better safety‑utility tradeoff on benchmarks such as AgentDojo.
arXiv:2608. 12851v1 Announce Type: new Abstract: Self-improving LLM agents convert successful trajectories into persistent cross-task state.
By Xutao Mao, Liangjie Zhao, Xiang Zheng, Cong Wang
arXiv:2608. 09885v1 Announce Type: new Abstract: The safety of large language model (LLM) agents depends not only on model weights but also on the agent harness that manages context, memory, tools, permissions, and runtime control.
By Wanying Qu, Qinghua Mao, Yu Li, Jiyao Liu, Xin Zhang, Dadi Guo, Yanxu Zhu, Qingyu Liu, Leitao Yuan, Xi Lin, Shanfeng Zhu, Yanwei Fu, Jing Shao, Xia Hu, Dongrui Liu
arXiv:2608. 02683v1 Announce Type: cross Abstract: Large Language Model (LLM) agents rely on multi-stage agentic workflows, with stages such as memory, planning, and tool execution, to accomplish complex tasks.
By Zibo Xiao, Haoyu Wang, Jun Sun
The paper investigates whether tool‑calling large language model agents maintain consistent safety throughout a conversation. It finds that agents are most vulnerable at the very start of a session, with safety improving significantly after completing a few regular agentic tasks—a phenomenon termed the cold‑start safety gap. The authors introduce the Safety Over Depth for Agents (SODA) benchmark to systematically study this effect, evaluate multiple models, and demonstrate that warming up agents with regular tasks before deployment enhances safety while preserving utility.
By Chung-En Sun, Linbo Liu, Tsui-Wei Weng
arXiv:2607. 19913v1 Announce Type: new Abstract: Agent safety is moving from content moderation toward preventing operational failures before tool-using agents act.
By Yuan Xiong, Linji Hao, Shizhu He, Yequan Wang, Lijun Li
arXiv:2607. 01793v1 Announce Type: new Abstract: LLM agents increasingly perform autonomous actions through external tools, leading to complex and evolving safety risks.
By Yunhao Feng, Ruixiao Lin, Ming Wen, Qinqin He, Yanming Guo, Yifan Ding, Yutao Wu, Jialuo Chen, Yunhao Chen, Xiaohu Du, Jianan Ma, Zixing Chen, Zhuoer Xu, Xingjun Ma, Xinhao Deng
HarnessRisk is a lifecycle-oriented benchmark for evaluating safety in agent harnesses that manage tools, extensions, state, permissions, and external actions. It defines six operational phases—Harness Configuration, Capability Extension, Runtime Operation, State Persistence, Action Control, and Incident Recovery—and includes 128 sandboxed cases pairing benign user objectives with adversarial instructions. Across three harnesses, six language models, and 14 configurations, attack success rates vary from 12.6% to 80.9%, with the most vulnerable phase being Harness Configuration.
"whyItMatters":"The benchmark demonstrates that safety failures can arise in multiple harness responsibilities and that even explicit risk detection does not guarantee safe action, underscoring the need for comprehensive evaluation across model and harness configurations."
By Yajing Bai, Jinhao Duan, Jie Peng, Xianfeng Wu, Sijia Liu, Song Wang, Tianlong Chen
arXiv:2607. 16247v1 Announce Type: new Abstract: While Vision-Language Models (VLMs) have empowered embodied agents to execute complex household tasks, they struggle to proactively handle dynamically emerging hazards during closed-loop interactions.
By Bingrui Sima, Lizhong Wang, Xiaoya Lu, Kun He, Xiao Yang
arXiv:2602. 03224v2 Announce Type: replace Abstract: Test-time evolution of agent memory represents a pivotal paradigm for advancing AGI, as it strengthens complex reasoning through experience accumulation without requiring parameter updates.
By Yu Cheng, Yongkang Hu, Jiuan Zhou, Yushuo Zhang, Yihang Chen, Huichi Zhou, Mingang Chen, Zhizhong Zhang, Kun Shao, Yuan Xie, Zhaoxia Yin