arXiv:2610.20111v1 Announce Type: new Abstract: Distilling pretrained video diffusion models into few-step autoregressive (AR) generators typically combines student initialization with distribution m...
arXiv:2609.36995v1 Announce Type: cross Abstract: Few-step streaming audio--video generation requires both causal modeling and step distillation, yet standard training recipes face two context-relate...
arXiv:2610.26025v2 Announce Type: replace-cross Abstract: Diffusion Transformers (DiTs) deliver high-fidelity image and video generation, but inference requires many sequential model evaluations. Two...
arXiv:2608. 01263v1 Announce Type: new Abstract: On-policy distillation (OPD) samples trajectories from the current student policy and minimizes token-level divergence between student and teacher next-token distributions at prefixes along those trajectories.
TAM (Task‑Aware Memory Distillation) is a framework that stores a frozen teacher’s knowledge in a bounded, retrievable memory, encoding latent features, forecast changes, or flow residuals. During training, the student matches the teacher’s similarity distribution over shared references or regresses observation‑conditioned residual prototypes, complementing supervised prediction and conventional distillation. Experiments on video prediction, weather forecasting, and traffic flow prediction show that TAM improves SSIM on all six video datasets and reduces MSE on five, achieving up to 1.93% MSE reduction on WeatherBench without affecting student inference cost.
We propose OPSD-V, an on-policy self-distillation paradigm for post-training few-step autoregressive (AR) video diffusion models. Existing few-step AR video generators can produce long videos with low latency, but still suffer from error accumulation and weakened motion dynamics during long autoregressive rollout.
arXiv:2605. 30116v2 Announce Type: replace-cross Abstract: Distribution Matching Distillation (DMD) is a widely used paradigm for accelerating inference in few-step video diffusion models.
The paper introduces VDOT++, a unified distillation framework for few-step video generation across text-to-video, image-to-video, and condition-based tasks. It extends the previous VDOT method by incorporating an asymmetric unbalanced optimal transport distillation that allows student tokens to carry less mass, and an α1 ground cost that uses a weighted median to preserve modes. The approach combines distribution matching with adversarial refinement and cross-scale distillation, achieving competitive performance on multiple video benchmarks with only four sampling steps.
On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules.
arXiv:2608. 03316v1 Announce Type: new Abstract: On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid.
DMAD (Distribution Matching as Adversarial Distillation) reinterprets distribution matching as a classification task, enabling a few‑step student model to learn log‑density ratios directly via two discriminator heads on a shared backbone. This eliminates the need for an auxiliary diffusion model, reducing memory and computation overhead. Experiments show DMAD achieves state‑of‑the‑art Fréchet Inception Distance scores on ImageNet‑64x64, COCO‑10K, and VBench, and outperforms competing few‑step methods in joint audio‑video generation on MiniMax‑H3‑33B.
arXiv:2609.35491v2 Announce Type: replace-cross Abstract: Few-step autoregressive video generation commonly relies on Distribution Matching Distillation (DMD), requiring a bidirectional diffusion tea...