The paper introduces a controlled evaluation set focused on the Go concepts of atari and sente, aiming to test whether language models can recognize these terms and link them to move evaluation. The authors fine‑tune a Go language model, LoGos, with terminology and commentary supervision and then assess its performance on local board changes and strategic move preferences. Results show that LoGos struggles to correctly identify atari relationships and to assess their strategic implications, yet its terminology predictions still influence move choices, sometimes contradicting the reference evaluation.
arXiv:2506. 17294v3 Announce Type: replace-cross Abstract: The advent of artificial intelligence has propelled AI-Generated Game Commentary (AI-GGC) into a rapidly expanding research area, offering advantages such as scalable availability and personalized narration.
arXiv:2404. 02039v5 Announce Type: replace Abstract: Game environments provide rich, controllable settings that stimulate many aspects of real-world complexity.
arXiv:2608. 07490v1 Announce Type: cross Abstract: Large language model agents are increasingly evaluated through games, but most benchmarks emphasize final outcomes rather than how players learn from repeated interaction.
GameCommBench is a unified benchmark for AI-generated game commentary that covers board games, sports, and esports, aligning commentary with diverse game contexts and annotating it by type. The authors introduce Type-Aware Commentary Evaluation (TACE), a structured framework that evaluates different commentary types, and validate its reliability and human agreement. Using TACE, they benchmark representative AI commentators and find that live observation and strategic analysis are major bottlenecks, revealing non-uniform capability profiles.
Dialogue games represent a challenging setting where complex cognitive skills are required to accomplish tasks while coordinating with other players. Considering that language represents an interface...
Game Arena is an open, continuously expanding platform that evaluates large language models through competitive games, allowing head‑to‑head matchups in structured environments. Unlike static benchmarks, it prevents performance saturation by increasing gameplay difficulty as models improve. The report outlines the infrastructure and presents three pilot games—Chess, Poker, and Werewolf—covering perfect information, imperfect information, and multiplayer settings, and details evaluation metrics and competition results.
The paper introduces an automatic metric for evaluating chess explanations based on chess-engine evaluations and finds that current language models perform poorly, relying on surface-level observations. It presents BinProp, a training-free method that helps models discover reusable chess concepts, significantly improving explainability across multiple models. BinProp also generalizes to other board games such as Go, draughts, and shogi without extra tuning.
arXiv:2608. 04240v1 Announce Type: cross Abstract: Superhuman game engines in domains like chess have made expert-level evaluations easily accessible, yet they communicate what is true without the natural-language explanations that make such expertise educationally useful to experts and non-experts alike.
arXiv:2509. 17192v3 Announce Type: replace Abstract: LLM-based social simulations can make a generated transcript look like a single behavioral signal, but the model behind that transcript may be doing several different jobs: choosing what an actor says or does, deciding what happens after an action, or both.
arXiv:2607. 27536v1 Announce Type: cross Abstract: Learning a strategic task changes more than what is directly taught: fine-tuning on one game can either enhance or degrade an agent's ability to reason in another.
The paper investigates whether large language models (LLMs) exhibit language‑specific skill differences by having two identical model instances play a text‑based game in different languages. Using a multilingual extension of TextArena, the authors evaluate three open‑weight models across eight languages and six games, finding that the same model can show markedly different performance—varying win–loss margins, invalid actions, and strategic choices—depending on the language interface. Analyses pinpoint language‑specific failures in spatial reasoning, card‑conditioned decisions, and optimal move selection, and demonstrate that adjusting the intermediate reasoning language can recover much of the lost performance.
The study reviews 377 academic papers on large language model (LLM)–based non‑player characters (NPCs) in video games, focusing on 42 key works. It quantitatively maps component adoption, noting that short‑term memory and world‑state grounding are common (60% and 69%), while long‑term memory, reflection, and retrieval‑augmented generation are less frequent. The paper also reports latency ranges (2.6–15 s) and identifies six structural barriers to commercial adoption, offering evidence‑based recommendations for developers and researchers.