MoI-Bench is a new benchmark designed for on‑device image generation and editing, featuring 64 subtasks and 1150 samples that cover 16 operations and eight visual target categories. It introduces MoI‑Eval, a task‑specific evaluation framework that assesses instruction following, visual quality, and detail preservation, and aggregates results into a two‑dimensional Quality‑Deployability map. The benchmark reveals that current on‑device models excel at efficient generation or editing but need improved editing quality and broader task coverage, especially for text replacement and identity preservation.
Recent advances in unified multimodal models have significantly improved text-guided image editing abilities. In particular, models such as Nano-Banana-Pro and GPT-Image-2 demonstrate emerging capabilities in multi-source image editing (MIE), including tasks such as object synthesis, person-background composition, and cross-image style fusion.
The paper introduces WeGenEditBench, a benchmark with 2,740 evaluation instances and 148 task types for image editing, focusing on local failure detection and prioritization for model improvement. It also presents WeGenEvoStudio, a multi‑agent harness that supports recursive self‑improvement by diagnosing failures, constructing retrieval‑grounded data, and adapting models. Experiments show that models such as GPT‑Image‑2 and Seedream‑5.0‑Pro excel in single‑image and text‑intensive editing, and evolution experiments demonstrate significant targeted gains on selected benchmarks.
arXiv:2606. 01213v1 Announce Type: cross Abstract: Despite tremendous recent progress, current text-guided image editing methods still struggle with many aspects of editing involving instruction following, minimally editing the source image, and ensuring high visual quality.
arXiv:2602.06663v3 Announce Type: replace Abstract: Unified multimodal models (UMMs) have shown impressive capabilities in complex natural image generation. However, their potential in computer-use p...
VGA‑BenchV2 is an expanded, human‑aligned benchmark and optimization framework that jointly evaluates video generation quality and aesthetic value. It builds on the original VGA‑Bench taxonomy, adding 52 sub‑dimensions and 1,016 curated prompts to generate over 60,000 videos from 12 mainstream models. The benchmark significantly enlarges human supervision with 36,000 task‑level annotations and introduces a hybrid evaluator (VAQA‑Net, VTag‑Net, VGQA‑Net) that aligns well with human judgments and can be used as a reward model for reinforcement‑learning fine‑tuning.
We introduce VGA-BenchV2, an extended human-aligned benchmark and optimization framework for jointly evaluating and improving video generation quality and aesthetic value. Built upon VGA-Bench, VGA-Be...
arXiv:2606. 19103v1 Announce Type: cross Abstract: Recent advances in instruction-based image editing have enabled models to perform complex visual edits from natural language instructions.
VTR-Bench is a new benchmark designed to evaluate how well video generation models render text within scenes. It includes 300 prompts across five real-world scenarios such as advertisements and scientific videos, and uses an automated pipeline with human alignment to assess text fidelity and scene/motion requirements. Experiments on 11 state‑of‑the‑art models show that even the best performer has a word error rate of 0.250, underscoring widespread challenges in visual text rendering.
arXiv:2610.17661v2 Announce Type: replace Abstract: The rapid development of mobile imaging and social platforms has led to explosive growth in user-generated content images, making their systematic...
OmniEdit-Bench introduces a comprehensive benchmark for instruction-based video editing (IVE), addressing limitations of existing datasets by covering spatial, temporal, audio, and reference-based editing tasks and distinguishing explicit from implicit instructions. The evaluation framework assesses editing quality across accuracy, preservation, realism, and consistency, using human judgments and vision-language models, and incorporates an accuracy-aware penalty to ensure instruction fidelity. Experiments reveal that current IVE models perform poorly, highlighting the need for improved methods.
arXiv:2602. 00122v3 Announce Type: replace-cross Abstract: In recent years, image editing models have made significant progress, enabling users to manipulate visual content in a flexible and interactive manner through natural language instructions.
arXiv:2608. 19637v1 Announce Type: new Abstract: Text editing in product posters entails inserting new text or replacing existing text while preserving product appearance, background content, and global composition.