Assessing Rule Adherence of LLM Adjudicators in Call of Cthulhu TRPG

The paper introduces CoC‑Seduce, a multi‑agent adversarial benchmark for evaluating how well large language models (LLMs) adhere to the rules of the tabletop role‑playing game Call of Cthulhu when acting as adjudicators. Using three LLMs to generate 5,376 samples across diverse settings and skill categories, the study benchmarks 22 target adjudicators and finds that newer releases and explicit reasoning do not guarantee robustness, with pseudo‑logic framing emerging as the most effective manipulation. The benchmark highlights the vulnerability of LLM adjudicators to rhetorical injection attacks that exploit narrative framing to bypass rule enforcement.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• agents
• benchmarks
• safety