arXiv AI

Evaluating Design Video Generation: Metrics for Compositional Fidelity

arXiv:2605. 16223v2 Announce Type: replace-cross Abstract: Generative video models are increasingly used in design animation tasks, yet no standardized evaluation framework exists for this domain.

arXiv AI
Jun 30

Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation

arXiv:2606. 28593v1 Announce Type: cross Abstract: While recent vision-language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs, it remains unclear whether they can recover temporal dynamics when motion is present.

By Anya Ji, Abhijith Varma Mudunuri, David M. Chan, Alane Suhr
arXiv Computer Vision
Aug 27

VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality

VGA‑BenchV2 is an expanded, human‑aligned benchmark and optimization framework that jointly evaluates video generation quality and aesthetic value. It builds on the original VGA‑Bench taxonomy, adding 52 sub‑dimensions and 1,016 curated prompts to generate over 60,000 videos from 12 mainstream models. The benchmark significantly enlarges human supervision with 36,000 task‑level annotations and introduces a hybrid evaluator (VAQA‑Net, VTag‑Net, VGQA‑Net) that aligns well with human judgments and can be used as a reward model for reinforcement‑learning fine‑tuning.

By Longteng Jiang, DanDan Zheng, Qianqian Qiao, Heng Huang, Huaye Wang, Yihang Bo, Bao Peng, Jingdong Chen, Jun Zhou, Xin Jin
arXiv Computer Vision
Sep 3

OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing

OmniEdit-Bench introduces a comprehensive benchmark for instruction-based video editing (IVE), addressing limitations of existing datasets by covering spatial, temporal, audio, and reference-based editing tasks and distinguishing explicit from implicit instructions. The evaluation framework assesses editing quality across accuracy, preservation, realism, and consistency, using human judgments and vision-language models, and incorporates an accuracy-aware penalty to ensure instruction fidelity. Experiments reveal that current IVE models perform poorly, highlighting the need for improved methods.

By Chenxuan Miao, Yutong Feng, Yi Lu, Yunfeng Yan, Donglian Qi, Shiwei Zhang, Yu Liu, Xi Chen, Hengshuang Zhao
arXiv AI
Jul 7

Motion Attribution for Video Generation

arXiv:2601. 08828v2 Announce Type: replace-cross Abstract: Despite the rapid progress of video generation models, the role of data in influencing motion is poorly understood.

By Xindi Wu, Despoina Paschalidou, Jun Gao, Antonio Torralba, Laura Leal-Taix\'e, Olga Russakovsky, Sanja Fidler, Jonathan Lorraine
Hugging Face Trending Papers
Aug 19

CamWorldQA: Perceptual Quality Assessment of Camera-Controlled World Video Generation

CamWorldQA introduces the first benchmark for assessing the perceptual quality of camera‑controlled world video generation, featuring 720 videos generated by six methods from 20 source videos across six camera trajectories, each scored by human raters. The paper also presents CWQA, a no‑reference quality assessment network that combines spatial, temporal motion, and optical flow features to predict quality scores. Experiments show CWQA outperforms existing VQA methods on the CamWorldQA dataset.