arXiv AI

Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation

arXiv Machine Learning
Jun 25

Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models

arXiv:2606. 25473v1 Announce Type: cross Abstract: Autoregressive video diffusion with causal diffusion transformers has emerged as a major paradigm for real-time streaming video generation and action-conditioned interactive world models.

By Kaiwen Zheng, Guande He, Min Zhao, Jintao Zhang, Huayu Chen, Jianfei Chen, Chen-Hsuan Lin, Ming-Yu Liu, Jun Zhu, Qianli Ma
arXiv Computer Vision
Oct 23

Direct Forcing: A Single-Stage Training Recipe for Few-Step Autoregressive Text-to-Video Generation

Direct Forcing is a single‑stage training framework for few‑step autoregressive text‑to‑video generation that eliminates the need for separate causal adaptation and distribution matching stages. It uses reference videos both as paired examples for causal continuation alignment and as a target distribution for rollout distribution matching, with dynamic gradient balancing to adjust their contributions during training. Starting from a bidirectional Wan2.1‑1.3B model, Direct Forcing trains a four‑step generator in only 250 updates, achieving state‑of‑the‑art VBench scores without additional initialization or diffusion‑score supervision.

By Haojun Xu, Jie Huang, Xin Lu, Mingchen Zhong, Zihao Fan, Linjiang Huang, Si Liu
arXiv Computer Vision
Sep 25

ViRDM: Taming Representation Distribution Matching for Few-Step Causal Video Generation

ViRDM is a new post‑training method for few‑step causal video generation that eliminates the need for a large teacher model and an online critic. By applying representation distribution matching (RDM) with a precomputed target distribution, a lightweight VAE decoder, and staged vector–Jacobian products, ViRDM overcomes memory, optimization, and temporal dynamics challenges. The approach reduces GPU memory usage and training time, achieving state‑of‑the‑art VBench performance with only 20 generator updates and 16 A100 GPU‑hours.

By Zichong Meng, Chongjian Ge, Chun-Hao P. Huang, Yang Zhou, Huaizu Jiang
arXiv AI
Oct 9

Conditional Residual Prediction: Improving Autoregressive Video Diffusion without a Bidirectional Teacher

The paper introduces Conditional Residual Prediction (CRP), a technique that reduces a causal video diffusion model’s dependence on past frames by having the model first predict the current frame without conditioning on history and then adding a residual based on the past. This approach allows training a causal model from an image‑model initialization without requiring a bidirectional teacher or complex distillation, and it nearly closes a 6.14‑point quality gap to a bidirectional model. Using CRP, the authors train Optica, a 2‑billion‑parameter causal video model that generates 5‑second 480p videos and achieves an 82.78 score on VBench with only about 15 million training videos.

By Bowen Zheng, Zhiguang Liu, Jiarong Ou, Rui Chen, Tianyang Hu