arXiv AI

AcrossVAM2.0: Conflict-Aware Point Supervision for Parameter-Efficient Video Diffusion Adaptation

arXiv Computer Vision
Oct 23

WorldForge V2: Eliciting Camera-Trajectory Alignment in Video Diffusion Models

WorldForge V2 introduces a training‑free guidance framework that refines video diffusion outputs along a target camera path using warped references and optical‑flow‑identified latent channels for selective guidance. It employs a dual‑path strategy to correct artifacts, a v‑cache to reduce refinement cost, and Trajectory LoRA adapters trained in two stages to focus on trajectory‑responsive network blocks. The approach achieves state‑of‑the‑art generation quality and trajectory accuracy with minimal training data, enabling reliable camera control across unseen scenes.

By Chenxi Song, Yanming Yang, Tong Zhao, Ruibo Li, Joey Tianyi Zhou, Chi Zhang
arXiv Computer Vision
Aug 24

Prompt2Effect: Training-Free Image-to-Video Model Specialization via LoRA Generation

arXiv:2606.13971v3 Announce Type: replace Abstract: While personalizing Image-to-Video (I2V) diffusion models with specific visual effects is increasingly demanded for high-end generation, current pr...

By Xiaomeng Yang, Yanyu Li, Gordon Guocheng Qian, Ivan Skorokhodov, Viacheslav Ivanov, Avalon Vinella, Xuan Zhang, Yanzhi Wang, Sergey Tulyakov, Anil Kag
arXiv Computer Vision
Sep 25

ViRDM: Taming Representation Distribution Matching for Few-Step Causal Video Generation

ViRDM is a new post‑training method for few‑step causal video generation that eliminates the need for a large teacher model and an online critic. By applying representation distribution matching (RDM) with a precomputed target distribution, a lightweight VAE decoder, and staged vector–Jacobian products, ViRDM overcomes memory, optimization, and temporal dynamics challenges. The approach reduces GPU memory usage and training time, achieving state‑of‑the‑art VBench performance with only 20 generator updates and 16 A100 GPU‑hours.

By Zichong Meng, Chongjian Ge, Chun-Hao P. Huang, Yang Zhou, Huaizu Jiang
arXiv AI
Aug 28

LiveVVT: High-Fidelity Video Virtual Try-On in Real Time

LiveVVT introduces a rolling streaming diffusion framework for video virtual try‑on that maintains high visual fidelity while enabling real‑time performance. It preserves bounded bidirectional modeling within a fixed‑size window, emits clean video chunks iteratively, and uses two memory modules—a bounded temporal memory and a persistent global appearance memory—to sustain long‑term consistency. A progressive distillation process further aligns teacher‑based bidirectional learning with causal few‑step inference, resulting in superior generation quality with 26× lower latency and 11× higher throughput compared to comparable models.

By Yushe Cao, Shikun Feng, Ruxiang Duan, Liyong Wang, Dianxi Shi, Chun Yu, Junliang Xing
arXiv AI
Jul 13

Transition Matching Distillation for Fast Video Generation

arXiv:2601. 09881v2 Announce Type: replace-cross Abstract: Large video diffusion and flow models have achieved remarkable success in high-quality video generation, but their use in real-time interactive applications remains limited due to their inefficient multi-step sampling process.

By Weili Nie, Julius Berner, Nanye Ma, Chao Liu, Saining Xie, Arash Vahdat
arXiv Computer Vision
Oct 23

Direct Forcing: A Single-Stage Training Recipe for Few-Step Autoregressive Text-to-Video Generation

Direct Forcing is a single‑stage training framework for few‑step autoregressive text‑to‑video generation that eliminates the need for separate causal adaptation and distribution matching stages. It uses reference videos both as paired examples for causal continuation alignment and as a target distribution for rollout distribution matching, with dynamic gradient balancing to adjust their contributions during training. Starting from a bidirectional Wan2.1‑1.3B model, Direct Forcing trains a four‑step generator in only 250 updates, achieving state‑of‑the‑art VBench scores without additional initialization or diffusion‑score supervision.

By Haojun Xu, Jie Huang, Xin Lu, Mingchen Zhong, Zihao Fan, Linjiang Huang, Si Liu
arXiv Computer Vision
Oct 16

FADRA: Frequency-Aware Diffusion with Residual Adaptation for Video Face Restoration

FADRA is a frequency‑aware diffusion framework designed for video face restoration (VFR). It combines a pre‑trained text‑to‑video diffusion model with lightweight LoRA adapters and a Low‑Quality Pixel‑Alignment Feature Fusion module to adapt the generative prior to VFR. The method introduces a Repeated Residual Adaptation Head that refines residuals step‑wise using low‑quality latent cues, and employs a Frequency‑Aware Loss to supervise multiple spectral bands, improving facial detail recovery and temporal consistency.

By Jin Jiang, Jia Wang, Panwen Hu, Weiran Zhao, Shengcai Liao
arXiv AI
Oct 15

LinMorph: Learning to Morph Weights for Efficient Video DiT Linearization

LinMorph proposes a learnable softmax‑to‑linear weight transformation for Video Diffusion Transformers, replacing traditional random initialization or manual weight reuse. By learning a shared morphism that maps each target Gated DeltaNet parameter to a combination of pretrained query, key, value, and output projections, it generates layer‑specific initializations that preserve pretrained knowledge. The method enables faster recovery and higher generation quality across pretrained Video DiTs, especially when applied to larger video scales after training on short, low‑resolution clips.

By Lingyi Long, Congyue Zhao, Xiaoou Li, Yanmin Qian, Tao Huang