arXiv AI
Oct 31

SemantiScene-Edit: Expert-Synthesized Supervision and Dual-Path Source Conditioning for Instruction-Based Video Editing

SemantiScene-Edit is a video editing system that uses a multimodal encoder and a pretrained video diffusion transformer to follow natural‑language instructions while preserving the source video’s appearance and temporal structure. The encoder processes source frames and the instruction together, sending semantic queries to the generator via a trainable connector and cross‑attention. A dual‑path visual route concatenates source‑video latents with noisy target latents to provide dense spatial and temporal information, and the model is trained with flow matching and a three‑stage curriculum on over 1.2 million instruction‑conditioned records, with a 100‑case benchmark for evaluation.

By Xinyu Chen, Song Wu, Dingbao Shao, Jiahang Li, Jiang Lin, Yuhang Liu, Duo Li, Jiaxin Hu, Shengrong Gu, Liang Li, Qian Wang, Junlan Feng, Ying Tai, Jian Yang, Zili Yi
arXiv AI
Oct 9

VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling

VINCIE‑NExT is a unified framework that transfers image editing capabilities to video editing by using in‑context visual demonstrations. It decomposes video editing into a chain of sub‑tasks (Video → Image → Image → Video), routing editing intent through the image domain and enabling joint training from heterogeneous image and video data under a single diffusion objective. The method introduces a novel position encoding to link image demonstrations and video frames, allowing pixel‑faithful appearance propagation, and supports test‑time scaling through progressive diffusion stages.

By Leigang Qu, Feng Cheng, Ziyan Yang, Bangbang Yang, Zhaoyang Huang, Wei Chow, Yicong Li, Wenjie Wang, Tat-Seng Chua, Yan Zeng
arXiv AI
Aug 26

TransPhy: Visual In-Context Learning for Physically Grounded Image Editing

TransPhy is a framework for visually in-context learning that focuses on physically grounded image editing. It introduces PhysVICL-74, a dataset of 74 transformation rules and 5,240 source–target pairs, and evaluates models on novel-instance transfer and unseen-rule generalization. The method predicts the demonstrated rule and a query-specific target-state description, then synthesizes the target image using token-wise mixture-of-experts guided by localized transition cues, improving rule adherence, query consistency, and generalization over existing methods.

By Siyi Xie, Xuanke Shi, Jinsheng Quan, Haoran Tang, Zukai Chen, Lei Yang, Quan Wang