Bidirectional Centered Score Matching (BCSM) is introduced as an efficient within-position distillation objective for discrete diffusion models, reducing the complexity from ≠|V|^2 to ≠K^2 by using an ≠K bidirectional active set. BCSM aligns relative logit geometry while maintaining a factorized ≠K objective, and it incorporates cross-timestep transition alignment. Experiments show significant efficiency gains, achieving up to 6× fewer neural function evaluations and notable improvements in HumanEval and MMMU metrics at single-step inference.
arXiv:2609.40235v1 Announce Type: cross Abstract: Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (...
The paper introduces Cumulative Marginal Matching (CMM), a new objective for training few‑step diffusion and flow‑matching models. CMM treats flow‑matching as a sequence of intermediate marginal distributions and directly optimizes the cumulative Wasserstein‑2 discrepancy between the base and few‑step models across the entire trajectory. It derives a tractable surrogate for this objective, shows that Shortcut‑CSL is a first‑order approximation of CMM, and reports improved generation quality on standard benchmarks under the same training budget.
arXiv:2509. 25837v3 Announce Type: replace-cross Abstract: Large language models (LLMs) deliver remarkable performance but are costly to deploy, motivating knowledge distillation (KD) for efficient inference.
The paper introduces VDOT++, a unified distillation framework for few-step video generation across text-to-video, image-to-video, and condition-based tasks. It extends the previous VDOT method by incorporating an asymmetric unbalanced optimal transport distillation that allows student tokens to carry less mass, and an α1 ground cost that uses a weighted median to preserve modes. The approach combines distribution matching with adversarial refinement and cross-scale distillation, achieving competitive performance on multiple video benchmarks with only four sampling steps.
Consistent Distribution Matching is a data‑free distillation technique that accelerates diffusion and flow models by unifying sample generation and score estimation in a single student network. The method requires only a frozen teacher and a trainable student, optimizing a single objective that provably leads to Wasserstein convergence of the student’s flow‑map to the teacher’s marginals. On ImageNet 256×256, it achieves an FID of 2.04 with one function evaluation and 1.37 with four evaluations after just 40 training epochs, outperforming existing data‑free baselines.
arXiv:2606. 05682v1 Announce Type: cross Abstract: Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments.
arXiv:2606. 05682v2 Announce Type: replace Abstract: Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments.
arXiv:2606. 29287v1 Announce Type: new Abstract: Diffusion and continuous-flow generative models achieve high-quality generation, and their deterministic sampling can be formulated as solving learned ODE dynamics.
arXiv:2602. 19066v2 Announce Type: replace-cross Abstract: Diffusion Language Models (DLMs) have recently achieved strong results in text generation.
arXiv:2607. 19686v1 Announce Type: cross Abstract: Masked diffusion models (MDMs) are a promising family of language generators, but achieving high-quality few-step generation remains challenging.
arXiv:2609.39657v1 Announce Type: new Abstract: Distilling pretrained foundation models into an autoencoder bottleneck improves latent diffusability, enabling diffusion models to converge faster and...
Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamic...