arXiv AI September 21, 2026 By Xiuhui Zhang, Yi Chen, Shusheng Xu, Fan Li, Huan Wang, Tongkai Yang, Binhang Yuan

GameASG-Bench: Benchmarking Autonomous Software Generation for Game Development

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv AI

2d ago

GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions

arXiv:2609.21562v1 Announce Type: cross Abstract: Coding agents can modify and test code across large software projects. Game development is a domain where agents must implement gameplay rules. A gam...

By Xinyu Che, Yunfei Ge, Shihao Li, Yanchen Liu, Hang Yan, Xinping Lei, Yanghai Wang, Zixuan Dong, Yifan Yao, Qianqian Xie, Letian Zhu, Jiaheng Liu

llms agents benchmarks

arXiv Computation and Language

2d ago

RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

arXiv:2609.22000v1 Announce Type: new Abstract: Computer-use agents (CUAs) have advanced along two separate lines: graphical interaction and software development through code and the command line. Re...

By Shuai Bai, Jiayong Deng, Yikun Fu, Chang Gao, Xuhao Hu, Mianqiu Huang, Yizhen Jiang, Yuheng Jing, Dehui Kong, Keliang Li, Ning Li, Wanli Li, Dayiheng Liu, Dunjie Lu, Changwei Luo, Que Shen, Zheyuan Wang, Zijian Wang, Jie Wu, Gao Wu, Zhihui Xie, Rui Xie, Haiyang Xu, An Yang, Jiakang Yuan, Yanming Zhang, Jiajun Zhang, Xi Zhang, Zhenru Zhang, Zhuo Zhen, Mingkang Zhu, Bowen Zhou

llms agents benchmarks

arXiv AI

Aug 25

GameXpert-Bench: How Far Are Coding Agents from Expert Game Development?

arXiv:2608.21833v1 Announce Type: new Abstract: Recent large language models (LLMs) can operate as coding agents that build complete games from natural language requests. Game development is especial...

By Kun Chen, Haorong Hong, Peizhong Gao, Jianfeng Lin, Tongxu Luo, Yuxuan Xie, Chenxu Liu, Jieling He, Zhongyuan Liu, Zeno Zeng

llms agents benchmarks

arXiv AI

Jul 20

Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios

arXiv:2604. 06742v2 Announce Type: replace-cross Abstract: The evolution of Large Language Models (LLMs) has catalyzed a paradigm shift towards intent-driven software development, where autonomous agents are expected to design and deliver complete, runnable software systems from scratch.

By Ruida Hu, Xinchen Wang, Chao Peng, Cuiyun Gao, David Lo

llms agents benchmarks

arXiv AI

Jun 9

SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?

arXiv:2606. 07682v1 Announce Type: cross Abstract: AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex environments.

By Rishi Desai, Jesse Hu, Joan Cabezas, Neel Harsola, Pratyush Shukla, Roey Ben Chaim, Adnan El Assadi, Omkaar Mukund Kamath, Fenil Faldu, Prannay Hebbar, Jiankai Sun, Yiyuan Li, Pramod Srinivasan, Ishan Gupta, Christopher Settles, Daniel Wang, Derek Chen, Pranav Raja, Albert Liu, Marek \v{S}uppa, Nevasini Sasikumar, Luyang Kong, Erik Quintanilla, Xiangyi Li, Ivan Bercovich, Steven Dillmann

agents benchmarks safety

arXiv AI

Jun 10

STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios

arXiv:2606. 10394v1 Announce Type: new Abstract: Large language models are increasingly used to power personal agents for everyday applications, but evaluating these agents remains a challenge.

By Sirui Liang, Bohan Yu, Peiyu Wang, Shiguang Guo, Wenxing Hu, Pengfei Cao, Jian Zhao, Cao Liu, Ke Zeng, Xunliang Cai, Kang Liu

llms agents benchmarks

One email a morning, machine-written

Related stories

GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions

RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

GameXpert-Bench: How Far Are Coding Agents from Expert Game Development?

Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios

SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?

STAGE-Claw: Automated State-based Agent Benchmarking for Realistic Scenarios