RefCraft is a new benchmark that tests AI agents on visual-reference-conditioned webpage development. It contains 300 realistic tasks where agents must combine natural‑language instructions with screenshots from one to five reference webpages, then render, inspect, and refine their outputs. The benchmark evaluates results with keypoints and programmatic/VLM judges, and the best current model scores only 52.12 out of 100, highlighting the difficulty of this capability.
By Yuchi Wang, Rongyao Fang, Weihao Yu, Xilin Wei, Tiancheng Gu, Dingkang Yang, Hang Zhang, Dayiheng Liu, Hongsheng Li, Shuai Bai
arXiv:2607. 10079v1 Announce Type: new Abstract: Digital Adoption Platforms (DAPs) are embedded overlays widely used on web systems to guide users through operations inside a page, helping them get started with unfamiliar interfaces quickly.
By Chengguang Gan, Hanjun Wei, Yunhao Liang, Zhixi Cai, Qinghao Zhang, Shiwen Ni
arXiv:2608. 12429v1 Announce Type: cross Abstract: Web agents often struggle to generalize to unseen websites because they lack website-specific supervision.
By Ruitao Wang, Yuwen Hao, Menglin Yang
arXiv:2601. 04203v2 Announce Type: replace-cross Abstract: We present FronTalk, a benchmark for front-end code generation that pioneers the study of a unique interaction dynamic: conversational code generation with multi-modal feedback.
By Xueqing Wu, Zihan Xue, Da Yin, Shuyan Zhou, Kai-Wei Chang, Nanyun Peng, Yeming Wen
arXiv:2606. 00154v1 Announce Type: cross Abstract: Recent advancements in multimodal large language models (MLLMs) have achieved remarkable progress in multimodal reasoning and code generation, catalyzing a new paradigm for front-end development.
By Fan Wu, Lishuai Dong, Cuiyun Gao, Yujia Chen, Yiming Huang, Yang Xiao, Qing Liao
arXiv:2511. 07332v2 Announce Type: replace-cross Abstract: Building reliable computer-use agents requires grounding: accurately connecting natural language instructions to the correct on-screen elements.
By Aarash Feizi, Shravan Nayak, Xiangru Jian, Kevin Qinghong Lin, Kaixin Li, Rabiul Awal, Xing Han L\`u, Johan Obando-Ceron, Juan A. Rodriguez, Nicolas Chapados, David Vazquez, Adriana Romero-Soriano, Reihaneh Rabbany, Perouz Taslakian, Christopher Pal, Spandana Gella, Sai Rajeswar
arXiv:2606. 17727v1 Announce Type: new Abstract: Recent vision-language models (VLMs) have shown promising progress in generating webpages from visual inputs, yet existing evaluations mainly focus on short, single-screen, and largely static webpages.
By Yi Zhao, Zhen Yang, Mengpan Chen, Mingde Xu, Shanghui Gong, Xijun Liu, Jibing Gong, Jie Tang
arXiv:2608. 14446v1 Announce Type: new Abstract: In the current artificial intelligence-driven innovation era, the pace of knowledge growth is accelerating, and is hard to keep up with.
By Beatrice Alessandra Motetti, Emilien Guandalino, Daniele Jahier Pagliari, Alessio Burrello, Lorenz K. M\"uller, Konstantin Berestizshevsky, Lukas Cavigelli
RAP (Reconstruct A Poster) is a new dataset and benchmark for generating conference posters from research papers, comprising over 25,000 verified paper‑poster pairs from 2022‑2026. Each pair is annotated with a three‑level content taxonomy and converted into styled HTML/CSS markup through an automated derendering pipeline, enabling objective render‑and‑compare evaluation. The corpus supports both inference and training, with a retrieval‑based PosterRAG pipeline and a supervised‑fine‑tuned model, and is publicly available via an online demo.
By Haonan Han, Yukang Lin, Minliang Lin, Haoxian Tan, Jun-Yan He, Rui Yang, Zhi-Qi Cheng, Jie Hu, Guanglu Wan, Xiu Li
arXiv:2601. 04126v3 Announce Type: replace-cross Abstract: GUI agents that interact with graphical interfaces on behalf of users represent a promising direction for practical AI assistants.
By Ziyun Zhang, Zezhou Wang, Xiaoyi Zhang, Zongyu Guo, Jiahao Li, Bin Li, Yan Lu
MOSS-Image is an open-weight image generation system that combines a 5B single-stream diffusion transformer with an intention-aware prompt enhancement module to produce high-fidelity, text-rich images with precise layout control. The system supports bilingual long-form text rendering and introduces evaluation benchmarks LongLongTextBench and RealLayoutArena. It achieves state‑of‑the‑art performance on LLTB and ranks highly on RLA, with plans to release the model, training recipe, and evaluation framework for the community.
By MOSI. AI
The paper introduces MAGE, a framework that uses user manuals to enhance computer use agents (CUAs) for new software. MAGE retrieves relevant manual content during execution, generates procedural guidance, and adapts the agent policy through interaction. In training, it converts manuals into executable task-plan pairs to guide rollouts, reducing the need for human‑annotated demonstrations and improving task success rates across six applications.
By Chulhoon Yoon, Kyungri Park, Jinyeong Yun, Daeyun Jeong, Sukmin Yun, Woohwan Jung