arXiv AI By Jiacheng Wang, Jinchang Hou, Fabian Wang, Ping Jian, Chenfu Bao, Zhonghou Lv

HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark

Read the original on arXiv AI →

HINTBench is a new benchmark for evaluating agents’ intrinsic risk, comprising 596 trajectories (400 synthetic risky, 136 synthetic safe, 30 real risky, 30 real safe) with an average length of 24 steps. It supports three tasks—risk detection, risk-step localization, and intrinsic failure-type identification—using a unified five-constraint taxonomy. Experiments show a large performance gap: while large language models can detect risky trajectories, they score below 37 on strict-F1 for risk-step localization, and existing guard models transfer poorly to this setting.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 21

ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis

arXiv:2604. 02022v4 Announce Type: replace Abstract: Evaluating the safety of LLM-based agents is increasingly important because risks in realistic deployments often emerge over multi-step interactions rather than isolated prompts or final responses.

By Yu Li, Haoyu Luo, Yuejin Xie, Yuqian Fu, Zhonghao Yang, Shuai Shao, Qihan Ren, Wanying Qu, Yanwei Fu, Yujiu Yang, Jing Shao, Xia Hu, Dongrui Liu
arXiv AI
Aug 7

DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model

arXiv:2608. 05695v1 Announce Type: new Abstract: As large language model (LLM) agents increasingly invoke external tools and interact with real-world systems, unsafe actions may cause irreversible consequences on external states, user data, and downstream services.

By Wenhao Lin, Chenyu Yu, Xingwei Lin, Sicong Cao, Xiang Chen, Lei Xue, Le Yu, Letian Sha, Chunming Wu
arXiv AI
Oct 9

Safe Actions Alone Do Not Ensure Safe Agents: Identifying Unfulfilled Obligations with Guard Models

Guard models are used to prevent LLM agents from performing forbidden actions, but this alone does not guarantee safety. The paper shows that many agent trajectories contain unfulfilled safety‑critical obligations—actions that should have been performed—highlighting a new source of risk. To address this, the authors introduce ObligationBench, a benchmark of 240 expert‑validated trajectories, and develop ObligationGuard, a model that improves obligation detection recall and exact‑match rates over existing models.

By Youwei Feng, Yitong Zhang, Yuetong Liu, Jia Li