arXiv AI

RefCraft: Benchmarking Visual-Reference-Conditioned Agentic Generation through Web Development

arXiv AI
Aug 25

ATP-Bench: Towards Agentic Tool Planning for MLLM Interleaved Generation

ATP‑Bench proposes a new benchmark for evaluating agentic tool planning in multimodal large language models (MLLMs) that generate interleaved text-and-image responses. The benchmark contains 7,702 QA pairs, including 1,592 visual‑question‑answer pairs, across eight categories and 25 visual‑critical intents, all verified by humans. A Multi‑Agent MLLM‑as‑a‑Judge (MAM) system is introduced to assess tool‑call precision, missed opportunities, and overall response quality without relying on ground‑truth references.

By Yinuo Liu, Zi Qian, Heng Zhou, Jiahao Zhang, Yajie Zhang, Zhihang Li, Mengyu Zhou, Erchao Zhao, Xiaoxi Jiang, Guanjun Jiang
arXiv Computer Vision
Sep 3

Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development

RILA is an execution‑driven agent that integrates browser rendering into the generation loop for interactive web development. It uses an Action Interaction Verification module to replay reference interactions on generated pages, collecting execution‑aware observations, and an Execution‑aware Rendering Score to jointly assess interaction correctness and visual fidelity during iterative optimization. A data synthesis pipeline further augments training data, enabling RILA to significantly improve interaction and visual quality across foundation models, even outperforming larger one‑shot generators.

By Yilong Guo, Hanqi Chen, Zixiao Ye, Guanzhong Wang, Chen Yu, Zeyu Chen
arXiv Machine Learning
Jun 11

FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback

arXiv:2601. 04203v2 Announce Type: replace-cross Abstract: We present FronTalk, a benchmark for front-end code generation that pioneers the study of a unique interaction dynamic: conversational code generation with multi-modal feedback.

By Xueqing Wu, Zihan Xue, Da Yin, Shuyan Zhou, Kai-Wei Chang, Nanyun Peng, Yeming Wen
arXiv AI
Sep 28

The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge

The paper introduces KNOWS, a benchmark for evaluating web agents that act as assistants by retrieving, synthesizing, and presenting information across complex, multi-step browser tasks. It outlines a task design rubric, evaluation protocol combining deterministic checks with LLM judgments, and reports that current agents achieve only modest success, with the best performing agent succeeding on less than 3% of tasks. The study highlights significant gaps in agents’ tool use, visual understanding, and long‑horizon reasoning.

By Alexander Gill, Md Farhan Ishmam, Xuyen Nguyen, Neha Bhat, Parker Henry DeYoung, Fateme Hashemi Chaleshtori, Nathan Stringham, Kenneth Marino, Ana Marasovi\'c
arXiv AI
Jun 4

Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation

arXiv:2605. 29861v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers, to deep research, which synthesizes scattered evidence into long-form reports.

By Chenghao Zhang, Guanting Dong, Yufan Liu, Tong Zhao, Xiaoxi Li, Zhicheng Dou
arXiv Computation and Language
Aug 31

LandingAgent: A Reference-Annotated Dataset and Agentic Generation Framework for Landing Pages

LandingAgent is a new framework for generating landing pages that are tailored to a specific target. It uses a reference‑annotated dataset called LandingBench, which abstracts real landing pages into structured elements such as section sequences, layout patterns, tone descriptors, visual emphasis, and CTA structure. The agentic framework operates in three phases—profiling the target, building a reference‑guided wireframe, and refining the page through critique—resulting in pages that are more faithful to the target, concise, readable, aesthetically pleasing, and structurally diverse compared to direct prompting.

By Injun Baek, HyeongSeok Lee, Yearim Kim, Junhoo Lee, Nojun Kwak
arXiv AI
Jun 17

LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings

arXiv:2606. 17727v1 Announce Type: new Abstract: Recent vision-language models (VLMs) have shown promising progress in generating webpages from visual inputs, yet existing evaluations mainly focus on short, single-screen, and largely static webpages.

By Yi Zhao, Zhen Yang, Mengpan Chen, Mingde Xu, Shanghui Gong, Xijun Liu, Jibing Gong, Jie Tang