arXiv Computer Vision September 3, 2026 By Yilong Guo, Hanqi Chen, Zixiao Ye, Guanzhong Wang, Chen Yu, Zeyu Chen

Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development

Read the original on arXiv Computer Vision →

RILA is an execution‑driven agent that integrates browser rendering into the generation loop for interactive web development. It uses an Action Interaction Verification module to replay reference interactions on generated pages, collecting execution‑aware observations, and an Execution‑aware Rendering Score to jointly assess interaction correctness and visual fidelity during iterative optimization. A data synthesis pipeline further augments training data, enabling RILA to significantly improve interaction and visual quality across foundation models, even outperforming larger one‑shot generators.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Jun 2

Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages

arXiv:2606. 00154v1 Announce Type: cross Abstract: Recent advancements in multimodal large language models (MLLMs) have achieved remarkable progress in multimodal reasoning and code generation, catalyzing a new paradigm for front-end development.

By Fan Wu, Lishuai Dong, Cuiyun Gao, Yujia Chen, Yiming Huang, Yang Xiao, Qing Liao

llms multimodal benchmarks

arXiv AI

Jul 15

1D-Bench: A Benchmark for Iterative UI Code Generation with Visual Feedback in Real-World

arXiv:2602. 18548v2 Announce Type: replace-cross Abstract: Design-to-code translates high-fidelity UI designs into executable front-end implementations, but progress remains hard to compare due to inconsistent datasets, toolchains, and evaluation protocols.

By Qiao Xu, Yipeng Yu, Chengxiao Feng, Xu Liu

reinforcement-learning multimodal benchmarks

arXiv AI

Jul 14

MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation

arXiv:2607. 10079v1 Announce Type: new Abstract: Digital Adoption Platforms (DAPs) are embedded overlays widely used on web systems to guide users through operations inside a page, helping them get started with unfamiliar interfaces quickly.

By Chengguang Gan, Hanjun Wei, Yunhao Liang, Zhixi Cai, Qinghao Zhang, Shiwen Ni

llms agents multimodal benchmarks

Hugging Face Trending Papers

Jul 7

UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex prompts that impose substantial demands on users and offer limited expressivity for page layout and cross-page visual coherence.

llms multimodal benchmarks

arXiv AI

Jul 8

UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

arXiv:2607. 06306v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated growing competence in web page generation.

By Grace Man Chen, Litao Guo, Yifan Wu, Yiyu Chen, Yenchi Tseng, Sicheng Liu, Yuyu Luo, Ying-Cong Chen

llms multimodal benchmarks

arXiv Machine Learning

Jun 11

FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback

arXiv:2601. 04203v2 Announce Type: replace-cross Abstract: We present FronTalk, a benchmark for front-end code generation that pioneers the study of a unique interaction dynamic: conversational code generation with multi-modal feedback.

By Xueqing Wu, Zihan Xue, Da Yin, Shuyan Zhou, Kai-Wei Chang, Nanyun Peng, Yeming Wen

llms agents multimodal benchmarks

One email a morning, machine-written

Related stories

Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages

1D-Bench: A Benchmark for Iterative UI Code Generation with Visual Feedback in Real-World

MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation

UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback