Towards a Unified Misuse Monitoring Benchmark
Read the original on arXiv AI →The paper introduces a unified framework for monitoring misuse of large language model agents in multi‑actor settings, focusing on two attack types: decomposition attacks and prompt injection attacks. It proposes tracking the agent’s externalised actions and determining whether the first detection of harm falls within a defined harm window. Using this formalism, the authors build a benchmark of about 6,200 conversation transcripts and evaluate 17 monitor configurations, finding that action‑framed monitors perform well on both threats while content‑framed monitors struggle with injection attacks, and highlighting limitations of traditional position‑blind metrics.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.