arXiv Computer Vision

Semantically Aligned Gradient-Driven Context-Preserving Image Editing

Semantically Aligned Gradient-Driven Context-Preserving Image Editing (IABEdit) is a model‑agnostic framework that embeds differentiable semantic verification into the training of generative image editors. By using a frozen vision‑language model to extract spatially‑aware descriptors from ground‑truth edits and a trainable aligner to reproduce them from generated outputs, the residual becomes a gradient that teaches the generator both what to edit and where, without adding inference‑time VLM cost. IABEdit is compatible with various backbones (e.g., U‑Net in Stable Diffusion and MMDiT in FLUX) and improves structural fidelity on MagicBrush, achieves state‑of‑the‑art instruction adherence on RealEdit and EMU Edit, and outperforms the proprietary Gemini agent on the D‑LORD surveillance benchmark under heavy occlusion. "whyItMatters":"IABEdit demonstrates that incorporating semantic verification during training can produce more accurate, well‑localized edits and outperform existing methods even in challenging surveillance scenarios, as shown by its superior metrics and human/GPT‑4o evaluations."

arXiv AI

Oct 9

VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling

VINCIE‑NExT is a unified framework that transfers image editing capabilities to video editing by using in‑context visual demonstrations. It decomposes video editing into a chain of sub‑tasks (Video → Image → Image → Video), routing editing intent through the image domain and enabling joint training from heterogeneous image and video data under a single diffusion objective. The method introduces a novel position encoding to link image demonstrations and video frames, allowing pixel‑faithful appearance propagation, and supports test‑time scaling through progressive diffusion stages.

By Leigang Qu, Feng Cheng, Ziyan Yang, Bangbang Yang, Zhaoyang Huang, Wei Chow, Yicong Li, Wenjie Wang, Tat-Seng Chua, Yan Zeng

diffusion benchmarks

arXiv AI

Jul 8

Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders

arXiv:2607. 06445v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly utilized as the conditioning backbone for diffusion-based image editing due to their remarkable multimodal reasoning capabilities.

By Yoav Baron, Sara Dorfman, Roni Paiss, Daniel Cohen-Or, Or Patashnik

llms diffusion multimodal

arXiv AI

Jun 26

Reconstruction Alignment Improves Unified Multimodal Models

arXiv:2509. 07295v4 Announce Type: replace-cross Abstract: Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture.

By Ji Xie, Trevor Darrell, Luke Zettlemoyer, XuDong Wang

rag diffusion multimodal benchmarks safety

arXiv Computer Vision

Aug 27

FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry

arXiv:2607.18227v2 Announce Type: replace Abstract: In line with the prevailing direction of vision research, we explore the integration of both generation and editing capabilities for video and imag...

By Dingyun Zhang, Lixue Gong, Wei Liu

computer-vision fine-tuning multimodal

arXiv AI

Oct 31

SemantiScene-Edit: Expert-Synthesized Supervision and Dual-Path Source Conditioning for Instruction-Based Video Editing

SemantiScene-Edit is a video editing system that uses a multimodal encoder and a pretrained video diffusion transformer to follow natural‑language instructions while preserving the source video’s appearance and temporal structure. The encoder processes source frames and the instruction together, sending semantic queries to the generator via a trainable connector and cross‑attention. A dual‑path visual route concatenates source‑video latents with noisy target latents to provide dense spatial and temporal information, and the model is trained with flow matching and a three‑stage curriculum on over 1.2 million instruction‑conditioned records, with a 100‑case benchmark for evaluation.

By Xinyu Chen, Song Wu, Dingbao Shao, Jiahang Li, Jiang Lin, Yuhang Liu, Duo Li, Jiaxin Hu, Shengrong Gu, Liang Li, Qian Wang, Junlan Feng, Ying Tai, Jian Yang, Zili Yi

llms rag diffusion multimodal benchmarks safety

arXiv AI

1d ago

WeLM-Image: A Unified Image Generation Model towards Visual Intelligence

WeLM-Image is a unified image foundation model that combines multimodal understanding with text‑to‑image generation, instruction‑guided editing, and multi‑reference synthesis. It uses a Mixture‑of‑Transformers backbone with Mixture‑of‑Experts modules, enabling 31.7B stored parameters and about 5.4B active generation parameters per latent token. The model, optimized for single‑GPU deployment, achieves top rankings on PRISM‑Bench, Qwen‑Image‑Bench, and UnicBench, and its turbo variant can generate a 1024×1024 image in under five seconds on a single NVIDIA H20 GPU.

By Canmiao Fu, Chen Li, Guohui Zhang, Hao Lu, Hongyang Wei, Huadong Wu, Huaisong Zhang, Hubery Yin, Liyuan Cui, Qixin Yan, Tong Shao, Xiaoqiang Zhou, Xinhang Leng, Ying Zhang, Zhenhai Liu, Zhipeng Huang

llms diffusion efficiency multimodal

arXiv Computer Vision

Oct 19

PreciseEdit: Strong Yet Stable Semantic Transport for One-Step Image Editing

PreciseEdit is a new one‑step image editing framework that improves semantic transport by applying strong target‑directed changes only along reliable directions and spatial locations. It is model‑agnostic, training‑free, and inversion‑free, and uses a dual‑noise‑scale strategy to balance strong editing with background fidelity and structural consistency. Experiments on PIE‑Bench show that PreciseEdit outperforms existing methods across a range of editing tasks.

By Jingyi Wang, Xiao Lu

diffusion benchmarks safety

arXiv Computer Vision

Aug 25

From Dense Prediction to Visual Editing: Structured Supervision for Unified Image and Video Creation

arXiv:2608.14740v2 Announce Type: replace Abstract: Unified image and video creation requires a model to follow diverse instructions while preserving identity, geometry, and temporal structure from v...

By Zhefan Rao, Bin Zou, Xuanhua He, Chong Hou Choi, Yanheng Li, Rui Liu, Haoxuan Che, Qifeng Chen

llms diffusion multimodal

arXiv AI

Jun 6

Edit-R2: Context-Aware Reinforcement Learning for Multi-Turn Image Editing

arXiv:2606. 05950v1 Announce Type: new Abstract: Text-guided image editing has advanced rapidly with diffusion models and unified multimodal foundation models.

By Yuxiao Ye, Haoran He, Fangyuan Kong, Xintao Wang, Pengfei Wan, Kun Gai, Ling Pan

diffusion reinforcement-learning multimodal benchmarks

arXiv AI

Sep 30

Representation by Design in Generation: Cross-View Class-Token Alignment in Diffusion Transformers

arXiv:2609.36348v1 Announce Type: cross Abstract: Generative and representation learning remain asymmetrically connected: semantic representations are used to improve diffusion generation, whereas th...

By Xiaoyu Wu, Yifei Wang, Chen Wei

llms diffusion computer-vision safety

arXiv AI

Sep 2

EdiTikZ: Scientific Figure Editing from Revision Trajectories

arXiv:2609.01409v1 Announce Type: new Abstract: Vision-language models (VLMs) have shown strong performance in generating scientific figures from text or images. However, producing publication-ready...

By Christian Greisinger, Zhixue Zhao, Steffen Eger

llms agents reinforcement-learning multimodal benchmarks

arXiv AI

Sep 3

Bernini: Latent Semantic Planning for Video Diffusion

Bernini proposes a unified framework that separates semantic planning and pixel rendering for video generation and editing. An MLLM-based planner predicts target semantics in ViT embedding space, while a DiT-based renderer synthesizes pixels conditioned on this plan, text features, and source VAE features for editing. The approach introduces Segment-Aware 3D Rotary Positional Embedding and chain-of-thought reasoning, achieving state‑of‑the‑art performance on diverse video benchmarks.

By Bernini Team, Chenchen Liu, Junyi Chen, Lei Li, Lu Chi, Mingzhen Sun, Zhuoying Li, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan

llms rag diffusion multimodal benchmarks

Related stories