arXiv AI
CraftBench-UE: Deterministic Evaluation for Coding Agents in Unreal Engine
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 9
WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
arXiv:2606. 09426v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools.
arXiv AI
Sep 21
GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions
arXiv:2609.21562v1 Announce Type: cross Abstract: Coding agents can modify and test code across large software projects. Game development is a domain where agents must implement gameplay rules. A gam...
arXiv AI
Sep 21
GameASG-Bench: Benchmarking Autonomous Software Generation for Game Development
arXiv:2609.21293v1 Announce Type: new Abstract: Autonomous software generation (ASG) aims to turn human requirements into executable applications, but delivering these applications does not necessari...
arXiv AI
Sep 18
BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software
BuildBench introduces a realistic benchmark for evaluating large language model agents on the task of compiling open‑source software (OSS). It includes diverse OSS projects that lack clear build instructions, have undocumented dependencies, and may require source patching or script modification. The authors also present OSS‑BUILD‑AGENT, a baseline LLM‑based agent that retrieves build instructions effectively and achieves state‑of‑the‑art performance on the benchmark.
arXiv AI
Aug 25
GameXpert-Bench: How Far Are Coding Agents from Expert Game Development?
arXiv:2608.21833v1 Announce Type: new Abstract: Recent large language models (LLMs) can operate as coding agents that build complete games from natural language requests. Game development is especial...
arXiv AI
Jun 9
SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?
arXiv:2606. 07682v1 Announce Type: cross Abstract: AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex environments.