arXiv AI
Oct 12

SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire

SpireBench evaluates long‑horizon strategic play by having large language models play the full commercial deck‑builder Slay the Spire, a game where early choices determine long‑term success. Across 20 LLMs at the highest difficulty, frontier models beat heuristic baselines and match average human performance but lag far behind experts, especially as runs lengthen. The benchmark reveals that strong models excel in early fights but misjudge risk over a run, and that their performance depends partly on prior game knowledge rather than on memories written during play.

By Scott C. Lowe, Sander W. Keemink