arXiv AI

VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents

arXiv:2605. 26144v2 Announce Type: replace-cross Abstract: We present VISTA (VIsual Spec-To-App Benchmark), a benchmark for evaluating the end-to-end web-app generation capabilities of LLM-based agents.

arXiv AI

Jul 8

UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

arXiv:2607. 06306v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated growing competence in web page generation.

By Grace Man Chen, Litao Guo, Yifan Wu, Yiyu Chen, Yenchi Tseng, Sicheng Liu, Yuyu Luo, Ying-Cong Chen

llms multimodal benchmarks

Hugging Face Trending Papers

Jul 7

UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex prompts that impose substantial demands on users and offer limited expressivity for page layout and cross-page visual coherence.

llms multimodal benchmarks

arXiv Machine Learning

Jun 11

FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback

arXiv:2601. 04203v2 Announce Type: replace-cross Abstract: We present FronTalk, a benchmark for front-end code generation that pioneers the study of a unique interaction dynamic: conversational code generation with multi-modal feedback.

By Xueqing Wu, Zihan Xue, Da Yin, Shuyan Zhou, Kai-Wei Chang, Nanyun Peng, Yeming Wen

llms agents multimodal benchmarks

arXiv Machine Learning

Jun 30

SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions

arXiv:2606. 30573v1 Announce Type: new Abstract: We introduce SWE-Interact, a new testbed for evaluating coding agents on multi-turn, interactive, user-driven software engineering tasks.

By Mohit Raghavendra, Anisha Gunjal, Aakash Sabharwal, Yunzhong He

llms agents benchmarks

arXiv AI

Aug 11

Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Models

arXiv:2608. 09666v1 Announce Type: new Abstract: Recent advances in visual generative models have enabled high-quality image and video generation, but evaluating these models often demands sampling hundreds or thousands of images or videos, which is computationally expensive.

By Shulin Tian, Ziqi Huang, Fan Zhang, Hongyuan Zhu, Yu Qiao, Ziwei Liu

agents diffusion benchmarks

arXiv AI

Jul 21

Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification

arXiv:2603. 26648v3 Announce Type: replace-cross Abstract: Recent advances in large language models have improved the capabilities of coding agents, yet systematic evaluation of complex, end-to-end website development remains limited.

By Zehai He, Wenyi Hong, Zhen Yang, Ziyang Pan, Mingdao Liu, Xiaotao Gu, Jie Tang

llms agents multimodal benchmarks

arXiv AI

Jun 17

LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings

arXiv:2606. 17727v1 Announce Type: new Abstract: Recent vision-language models (VLMs) have shown promising progress in generating webpages from visual inputs, yet existing evaluations mainly focus on short, single-screen, and largely static webpages.

By Yi Zhao, Zhen Yang, Mengpan Chen, Mingde Xu, Shanghui Gong, Xijun Liu, Jibing Gong, Jie Tang

llms agents multimodal benchmarks

arXiv AI

Jun 3

VeRO: A Harness for Agents to Optimize Agents

arXiv:2602. 22480v4 Announce Type: replace Abstract: An important emerging application of coding agents is agent harness optimization: the iterative improvement of a target agent by editing and evaluating its code.

By Varun Ursekar, Apaar Shanker, Veronica Chatrath, Yuan Xue, Samuel Marc Denton

llms agents benchmarks

arXiv AI

Jul 14

MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation

arXiv:2607. 10079v1 Announce Type: new Abstract: Digital Adoption Platforms (DAPs) are embedded overlays widely used on web systems to guide users through operations inside a page, helping them get started with unfamiliar interfaces quickly.

By Chengguang Gan, Hanjun Wei, Yunhao Liang, Zhixi Cai, Qinghao Zhang, Shiwen Ni

llms agents multimodal benchmarks

arXiv AI

Jun 2

3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code

arXiv:2606. 01057v1 Announce Type: cross Abstract: Procedural 3D modeling through code is emerging as a versatile paradigm, offering deterministic, engine-ready, and precisely editable assets that neural 3D generators inherently lack.

By Yipeng Gao, Lei Shu, Genzhi Ye, Xi Xiong, Ameesh Makadia, Meiqi Guo, Laurent Itti, Jindong Chen

llms agents multimodal benchmarks

arXiv AI

Jun 16

Imperfect Visual Verification for Code Edition : A Case Study on TikZ

arXiv:2606. 15693v1 Announce Type: cross Abstract: LLMs have significantly advanced code generation, enabling the synthesis of functional programs.

By Charly Reux (UR, INSA Rennes, DiverSe), Mathieu Acher (CNRS, IUF, IRISA, UR, DiverSe), Djamel Eddine Khelladi (DiverSe, UR, CNRS, IRISA), Cl\'ement Quinton (SPIRALS, CNRS), Olivier Barais (UR, IRISA, DiverSe)

llms benchmarks

arXiv Machine Learning

Jul 30

REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage

arXiv:2604. 01527v4 Announce Type: replace-cross Abstract: Production deployment of AI coding agents requires fast, reproducible evaluation signals.

By Smriti Jha, Matteo Paltenghi, Chandra Maddila, Vijayaraghavan Murali, Shubham Ugare, Satish Chandra

llms agents benchmarks

Related stories