Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Recent advances in unified multimodal models have significantly improved text-guided image editing abilities. In particular, models such as Nano-Banana-Pro and GPT-Image-2 demonstrate emerging capabilities in multi-source image editing (MIE), including tasks such as object synthesis, person-background composition, and cross-image style fusion.
arXiv:2606. 08016v1 Announce Type: cross Abstract: Current image editing software often hinges on fixed filters or expert tuning, leaving a gap between amateur users' intent and outcomes.
MR‑IQA‑2 introduces an actor‑editor‑judge framework that separates reasoning from rating in image quality assessment. The actor generates quality reasoning, the editor modifies the image based on identified factors, and a frozen judge compares the original and edited images to provide reflective supervision. Fine‑grained credit assignment allows distinct supervision signals for reasoning and rating, achieving competitive human‑aligned ratings while offering richer, faithful visual understanding.
The paper introduces WaterBird-Edit, a benchmark that separates bird habitat labels from background context to detect evidence misgrounding in multimodal knowledge editing. It proposes VICE, a retrieval‑based framework that stores edits in external memory and uses visual in‑context demonstrations to focus on task‑relevant evidence. Experiments on three benchmarks show VICE improves lifelong editing performance and reduces sensitivity to background changes on WaterBird-Edit.
EditAgent rethinks agentic image editing by expanding both the action and observation spaces. It routes each editing step to general editors and specialized methods, attaching verification records that allow off‑the‑shelf VLM judges to outperform finetuned reward models. The approach yields parallel candidate generation and a single listwise judgment, which experts prefer over sequential refinement in 78.5% of cases and improves two open‑weight editors across three benchmarks.
The paper introduces WeGenEditBench, a benchmark with 2,740 evaluation instances and 148 task types for image editing, focusing on local failure detection and prioritization for model improvement. It also presents WeGenEvoStudio, a multi‑agent harness that supports recursive self‑improvement by diagnosing failures, constructing retrieval‑grounded data, and adapting models. Experiments show that models such as GPT‑Image‑2 and Seedream‑5.0‑Pro excel in single‑image and text‑intensive editing, and evolution experiments demonstrate significant targeted gains on selected benchmarks.