arXiv Computer Vision

PrismGPT: Proxy-Guided Learning for Region-Aware Photo Editing with Self-Synthesized Reasoning

PrismGPT is a Vision‑Language Model that generates structured, region‑aware photo‑editing plans from a single image, without relying on commercial black‑box tools. It learns to diagnose aesthetic issues globally and locally while predicting precise editing parameters, using proxy‑guided learning with operation decomposition and region‑aware aesthetic ranking to bootstrap the model. A competence‑based dynamic scheduler shifts training focus from proxy tasks to the main editing task as skills improve, and all reasoning traces for fine‑tuning are self‑synthesized by the model itself. Experiments on MIT‑Adobe FiveK and a new professionally retouched benchmark, SPIRE, show PrismGPT achieves state‑of‑the‑art results using only about 6% of the training data required by previous methods.

Hugging Face Trending Papers

Jun 1

MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching

Recent breakthroughs in instruction-based image editing have captured significant attention, as models are now capable of handling real-world editing demands with the practicality required by everyday users. However, editing models trained primarily for single-turn edits often break down in multi-turn editing--the natural interactive setting where a user iteratively refines an image based on the model's own previous outputs.

diffusion reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 3

Towards Robust Sequential Decomposition for Complex Image Editing

arXiv:2605. 09233v2 Announce Type: replace-cross Abstract: Recent advances in visual generative models have enabled high-fidelity image editing guided by human instructions.

By Zilai Zeng, Mingdeng Cao, Zijie Li, Xiaochen Lian, Yichun Shi, Peihao Zhu, Chen Sun, Peng Wang

robotics fine-tuning data-engineering

arXiv AI

Aug 18

OpenGPT-4o-Image: A Comprehensive Dataset for Advanced Image Generation and Editing

arXiv:2509. 24900v2 Announce Type: replace-cross Abstract: The performance of unified multimodal models for image generation and editing is fundamentally constrained by the quality and comprehensiveness of their training data.

By Zhihong Chen, Xuehai Bai, Yang Shi, Chaoyou Fu, Huanyu Zhang, Haotian Wang, Xiaoyan Sun, Zhang Zhang, Liang Wang, Yuanxing Zhang, Pengfei Wan, Yi-Fan Zhang

llms diffusion robotics fine-tuning multimodal benchmarks

arXiv AI

Aug 21

DARS: Dual-Level Credit Assignment RL with Structured Reasoning for Instruction-Based Image Editing

arXiv:2608. 20161v1 Announce Type: new Abstract: Instruction-based image editing uses a planner-renderer pipeline: a vision-language model (VLM) first converts the instruction into an edit plan, and a diffusion model then executes that plan.

By Haoxiang Cao, Jiajiong Cao, Xuanpu Zhang, Changqian Yu, Chaoqun Wang

llms diffusion reinforcement-learning multimodal benchmarks

arXiv Computer Vision

3d ago

ThinkV2V: Unleashing the Reasoning Capability of MLLMs for Instruction-Guided Video Editing

arXiv:2609.38541v1 Announce Type: new Abstract: Instruction-guided video editing has made significant progress, yet existing methods use multimodal large language models (MLLMs) primarily as semantic...

By Donghao Zhou, Haoyang He, Fan Zhang, Hao Yang, Guisheng Liu, Xin Gao, Zhongwei Wan, Xingyuan Bu, Jie Wang, Qiangpeng Yang, Shilei Wen, Chi-Wing Fu, Pheng-Ann Heng

llms multimodal benchmarks

arXiv AI

Jun 2

CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferences

arXiv:2606. 00931v1 Announce Type: cross Abstract: Instruction-guided image editing is becoming a general interface for visual work, yet existing benchmarks still focus largely on narrow appearance edits and do not fully capture the diversity of real-image tasks in professional workflows.

By Fangzhou Lin, Peiran Li, Lingyu Xu, Wenjing Chen, Qianwen Ge, Shuo Xing, Mingyang Wu, Xiangbo Gao, Siyuan Yang, Kazunori Yamada, Ziming Zhang, Haichong Zhang, Zhen Dong, Ming-Hsuan Yang, Zhengzhong Tu

agents computer-vision multimodal benchmarks

arXiv AI

Sep 3

Bernini: Latent Semantic Planning for Video Diffusion

Bernini proposes a unified framework that separates semantic planning and pixel rendering for video generation and editing. An MLLM-based planner predicts target semantics in ViT embedding space, while a DiT-based renderer synthesizes pixels conditioned on this plan, text features, and source VAE features for editing. The approach introduces Segment-Aware 3D Rotary Positional Embedding and chain-of-thought reasoning, achieving state‑of‑the‑art performance on diverse video benchmarks.

By Bernini Team, Chenchen Liu, Junyi Chen, Lei Li, Lu Chi, Mingzhen Sun, Zhuoying Li, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan

llms rag diffusion multimodal benchmarks

arXiv AI

Jun 6

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

arXiv:2606. 05950v1 Announce Type: new Abstract: Text-guided image editing has advanced rapidly with diffusion models and unified multimodal foundation models.

By Yuxiao Ye, Haoran He, Fangyuan Kong, Xintao Wang, Pengfei Wan, Kun Gai, Ling Pan

diffusion reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jun 2

PaintBench: Deterministic Evaluation of Precise Visual Editing

arXiv:2606. 00188v1 Announce Type: cross Abstract: While current multimodal models are proficient at open-ended visual editing, executing precise single-answer edits remains an important obstacle.

By Kai Xu, Ellis Brown, Shrikar Madhu, Rob Fergus, He He, Saining Xie

robotics multimodal benchmarks safety

arXiv Computer Vision

Aug 27

FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry

arXiv:2607.18227v2 Announce Type: replace Abstract: In line with the prevailing direction of vision research, we explore the integration of both generation and editing capabilities for video and imag...

By Dingyun Zhang, Lixue Gong, Wei Liu

computer-vision fine-tuning multimodal

Hugging Face Trending Papers

Jul 21

ExpertVerse: A General-Purpose Benchmark for Expert-Level Reasoning in Knowledge-Intensive Visual Synthesis

Recent advances in multimodal generative models have enabled instruction-based image generation to move beyond semantic manipulation to knowledge-driven visual reasoning. However, these methods focus on explicit commonsense reasoning, shallow causal understanding, and direct knowledge recall, failing at knowledge-intensive generation.

diffusion reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 9

IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment

arXiv:2606. 08016v1 Announce Type: cross Abstract: Current image editing software often hinges on fixed filters or expert tuning, leaving a gap between amateur users' intent and outcomes.

By Zichen Zhu, Yuheng Sun, Mingxuan Zhu, Wenjie Ma, Situo Zhang, Zhexiang Wang, Ziyue Yang, Danyang Zhang, Kunyao Lan, Zihan Zhao, Dingye Liu, Siqi Xiang, Lu Chen, Kai Yu

agents nlp fine-tuning safety

Related stories