arXiv Computation and Language By Taiyi Chen, Iskar Deng, Amandalynne Paullada

Recognizing Atari and Assessing Sente: Controlled Evaluation Tasks for Language-Based Go Agents

Read the original on arXiv Computation and Language →

The paper introduces a controlled evaluation set focused on the Go concepts of atari and sente, aiming to test whether language models can recognize these terms and link them to move evaluation. The authors fine‑tune a Go language model, LoGos, with terminology and commentary supervision and then assess its performance on local board changes and strategic move preferences. Results show that LoGos struggles to correctly identify atari relationships and to assess their strategic implications, yet its terminology predictions still influence move choices, sometimes contradicting the reference evaluation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
Jul 1

From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary

arXiv:2506. 17294v3 Announce Type: replace-cross Abstract: The advent of artificial intelligence has propelled AI-Generated Game Commentary (AI-GGC) into a rapidly expanding research area, offering advantages such as scalable availability and personalized narration.

By Qirui Zheng, Xingbo Wang, Keyuan Cheng, Yunlong Lu, Muhammad Asif Ali, Lingfeng Li, Yongyi Wang, Wenxin Li
arXiv AI
Oct 9

GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary

GameCommBench is a unified benchmark for AI-generated game commentary that covers board games, sports, and esports, aligning commentary with diverse game contexts and annotating it by type. The authors introduce Type-Aware Commentary Evaluation (TACE), a structured framework that evaluates different commentary types, and validate its reliability and human agreement. Using TACE, they benchmark representative AI commentators and find that live observation and strategic analysis are major bottlenecks, revealing non-uniform capability profiles.

By Qirui Zheng, Zhengteng Lin, Yunyi Xiao, Junhao Li, Keyuan Cheng, Xingbo Wang, Yongyi Wang, Lingfeng Li, Yunlong Lu, Wenxin Li
arXiv AI
Sep 28

Game Arena: Strategic LLM Evaluation in Competitive Environments

Game Arena is an open, continuously expanding platform that evaluates large language models through competitive games, allowing head‑to‑head matchups in structured environments. Unlike static benchmarks, it prevents performance saturation by increasing gameplay difficulty as models improve. The report outlines the infrastructure and presents three pilot games—Chess, Poker, and Werewolf—covering perfect information, imperfect information, and multiplayer settings, and details evaluation metrics and competition results.

By Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu, Hann Wang, Timothy Chung, Martyna Plomecka, John Schultz, Jon Lipovetz, Clayton Drazner, Yuchen Zhuang, Jaimie Hwang, Nate Keating, Riley Jones, Andrew Lee, Oran Kelly, Ian Gemp, Michael Aaron, Laurel Prince, Kate Larson, Jeff Moser, Harrison Jobe, Chad Woodford, Siqi Liu, Andrew Wang, Bo Chang, Christopher D'Mello, Diane Chaleff, Addison Howard, Johnny Yip, Chuck Sugnet, Antonio Gulli, Meghan O'Connell, Will Cukierski, Nenad Tomasev, Dima Yeroshenko, Kinjal Parekh, Roxanne Daniel, Marc Lanctot, Domino Weir, Elsa Dong, Daniel Hennes, Melissa Nalubwama, Robert Fraser, Ryan Trostle, Jun Peng, Tom Mason, Lloyd Hightower, Chiamaka Chukwuka, Yuexiang Zhai, Phoebe Kirk, Yi Su, Yuting Han, Jie Ren, Chris Prichard, Sahand Sharifzadeh, Karim Hakimzadeh, DJ Sterling, Meg Risdal, Kate Olszewska, Ya Xu, Orhan Firat, Minmin Chen