arXiv AI
Oct 31

SemantiScene-Edit: Expert-Synthesized Supervision and Dual-Path Source Conditioning for Instruction-Based Video Editing

SemantiScene-Edit is a video editing system that uses a multimodal encoder and a pretrained video diffusion transformer to follow natural‑language instructions while preserving the source video’s appearance and temporal structure. The encoder processes source frames and the instruction together, sending semantic queries to the generator via a trainable connector and cross‑attention. A dual‑path visual route concatenates source‑video latents with noisy target latents to provide dense spatial and temporal information, and the model is trained with flow matching and a three‑stage curriculum on over 1.2 million instruction‑conditioned records, with a 100‑case benchmark for evaluation.

By Xinyu Chen, Song Wu, Dingbao Shao, Jiahang Li, Jiang Lin, Yuhang Liu, Duo Li, Jiaxin Hu, Shengrong Gu, Liang Li, Qian Wang, Junlan Feng, Ying Tai, Jian Yang, Zili Yi
arXiv AI
Oct 9

VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling

VINCIE‑NExT is a unified framework that transfers image editing capabilities to video editing by using in‑context visual demonstrations. It decomposes video editing into a chain of sub‑tasks (Video → Image → Image → Video), routing editing intent through the image domain and enabling joint training from heterogeneous image and video data under a single diffusion objective. The method introduces a novel position encoding to link image demonstrations and video frames, allowing pixel‑faithful appearance propagation, and supports test‑time scaling through progressive diffusion stages.

By Leigang Qu, Feng Cheng, Ziyan Yang, Bangbang Yang, Zhaoyang Huang, Wei Chow, Yicong Li, Wenjie Wang, Tat-Seng Chua, Yan Zeng