arXiv Computer Vision

TripleFlow: Training-Free Video Object Removal by Bridging Residual Editing and Native Generation

arXiv AI
Oct 31

SemantiScene-Edit: Expert-Synthesized Supervision and Dual-Path Source Conditioning for Instruction-Based Video Editing

SemantiScene-Edit is a video editing system that uses a multimodal encoder and a pretrained video diffusion transformer to follow natural‑language instructions while preserving the source video’s appearance and temporal structure. The encoder processes source frames and the instruction together, sending semantic queries to the generator via a trainable connector and cross‑attention. A dual‑path visual route concatenates source‑video latents with noisy target latents to provide dense spatial and temporal information, and the model is trained with flow matching and a three‑stage curriculum on over 1.2 million instruction‑conditioned records, with a 100‑case benchmark for evaluation.

By Xinyu Chen, Song Wu, Dingbao Shao, Jiahang Li, Jiang Lin, Yuhang Liu, Duo Li, Jiaxin Hu, Shengrong Gu, Liang Li, Qian Wang, Junlan Feng, Ying Tai, Jian Yang, Zili Yi
arXiv AI
Oct 9

VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling

VINCIE‑NExT is a unified framework that transfers image editing capabilities to video editing by using in‑context visual demonstrations. It decomposes video editing into a chain of sub‑tasks (Video → Image → Image → Video), routing editing intent through the image domain and enabling joint training from heterogeneous image and video data under a single diffusion objective. The method introduces a novel position encoding to link image demonstrations and video frames, allowing pixel‑faithful appearance propagation, and supports test‑time scaling through progressive diffusion stages.

By Leigang Qu, Feng Cheng, Ziyan Yang, Bangbang Yang, Zhaoyang Huang, Wei Chow, Yicong Li, Wenjie Wang, Tat-Seng Chua, Yan Zeng
arXiv Computer Vision
Oct 23

UniVOE: A Removal-Driven Framework for Interaction-Aware Universal Video Object Editing

UniVOE is a removal‑driven framework that trains both video object removal and injection models. It first builds an effect‑aware remover by leveraging video inpainting priors and fine‑tuning on object‑present/object‑absent pairs, then uses the remover to generate large‑scale paired data for training an injection model that restores objects with their dynamics and interaction effects. A reward‑based post‑training step aligns injection results with human preferences, and experiments show UniVOE outperforms existing open‑source and commercial systems, especially in challenging interaction‑effect scenarios.

By Shuzhou Yang, Lanqing Guo, Lingen Li, Weiqi Li, Zhao Zhong, Liefeng Bo, Jian Zhang
arXiv Computer Vision
Oct 9

From Video Clips to Creation Trajectory: Sora100K for AI-Native Video Creation

The paper introduces Sora100K, a dataset that captures AI‑Native video creation workflows as structured trajectories rather than isolated clips. It categorizes trajectories into text‑to‑video generation roots, single‑turn editing edges, and multi‑turn editing sequences, annotating each with semantic and operation labels via a VLM. A rigorous construction pipeline ensures accurate source‑to‑edit lineage and intermediate states, and lightweight adaptation on LTX‑2 models demonstrates improved visual quality, multi‑shot generation, and cross‑shot consistency, though multi‑turn editing remains challenging.

By Sicong Yang, Ruihuan Yang, Jian Lu, Jianfei Yuan, Xiaodong Cun, Xiuli Bi
arXiv Computer Vision
Nov 2

Sparkle: Realizing Lively Instruction-Guided Video Background Replacement via Decoupled Guidance

Sparkle introduces a new dataset of approximately 140,000 video pairs for instruction‑guided background replacement, covering five common themes. The authors present a scalable pipeline that decouples foreground and background guidance with strict quality filtering, addressing the lack of precise background cues in existing data. Experiments show that models trained on Sparkle outperform all current baselines on both the OpenVE-Bench and the newly released Sparkle-Bench.

By Ziyun Zeng, Yiqi Lin, Guoqiang Liang, Mike Zheng Shou