arXiv AI
GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 25
GameXpert-Bench: How Far Are Coding Agents from Expert Game Development?
arXiv:2608.21833v1 Announce Type: new Abstract: Recent large language models (LLMs) can operate as coding agents that build complete games from natural language requests. Game development is especial...
arXiv AI
2d ago
GameASG-Bench: Benchmarking Autonomous Software Generation for Game Development
arXiv:2609.21293v1 Announce Type: new Abstract: Autonomous software generation (ASG) aims to turn human requirements into executable applications, but delivering these applications does not necessari...
arXiv AI
Jul 3
TestEvo-Bench: An Executable and Live Benchmark for Test and Code Co-Evolution
arXiv:2607. 02469v1 Announce Type: cross Abstract: Software tests and code evolve together: a code change should be followed by new or updated tests that record the new software behavior.
arXiv Computation and Language
Sep 10
SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
arXiv:2605.21384v2 Announce Type: replace-cross Abstract: As long-horizon coding agents produce more code than any developer can review, oversight collapses onto a single surface: the automated test...
arXiv AI
Jun 9
Executable World Models for ARC-AGI-3 in the Era of Coding Agents
arXiv:2605. 05138v2 Announce Type: replace Abstract: We evaluate an initial coding-agent system for ARC-AGI-3 in which the agent maintains an executable Python world model, verifies it against previous observations, refactors it toward simpler abstractions as a practical proxy for an MDL-like simplicity bias, and plans through the model before acting.
arXiv AI
Aug 11
A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents
arXiv:2608. 09072v1 Announce Type: cross Abstract: Large language model-powered coding agents are increasingly used to modify existing code repositories, for example, by adding features or fixing bugs.