The paper introduces SVDtrunc, a two‑step block‑level compression method for Diffusion Transformers (DiTs) that allocates ranks across blocks, applies truncated SVD to the least important ones, and then fine‑tunes all blocks with modular knowledge distillation and a rectified‑flow objective. Experiments on FLUX.dev show that SVDtrunc achieves near‑full performance at 68% of the original parameters and remains competitive even at 57%, outperforming all competing approaches on GenEval, HPSv2, and DPG benchmarks. The method also works well without fine‑tuning, complementing step distillation and offering a practical path to efficient large‑scale generative models.
arXiv:2608. 01298v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) have emerged as a core architecture in generative modeling due to their scalability and adaptability to multimodal tasks.
arXiv:2605. 26632v2 Announce Type: replace Abstract: Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs.
arXiv:2605. 20708v2 Announce Type: replace-cross Abstract: Diffusion Transformers (DiTs) have become a de facto backbone of modern visual generation, and nearly every major axis of their design -- tokenization, attention, conditioning, objectives, and latent autoencoders -- has been extensively revisited.
arXiv:2605. 26632v3 Announce Type: replace Abstract: Diffusion Transformers (DiT) achieve strong performance in image generation but incur substantial inference costs.
arXiv:2607. 00927v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) have demonstrated impressive performance in image generation but suffer from substantial computational overhead and resource consumption.
arXiv:2604. 16514v5 Announce Type: replace-cross Abstract: Autoregressive vision-language models (VLMs) deliver strong multimodal capability, but their token-by-token decoding imposes a fundamental inference bottleneck.
arXiv:2607. 24665v1 Announce Type: cross Abstract: Modern large language models scale successfully by pairing capacity growth with efficiency, keeping per-token and deployment costs under control as capacity grows.
LLaDA-Image is a unified framework that couples a 6B Diffusion Transformer (DiT) trained from scratch with a frozen vision‑language module based on the LLaDA2.0‑Mini diffusion language model. The approach first builds a strong visual generative prior through image‑only pre‑training and mid‑training, then fine‑tunes with a 220M‑sample generation pipeline that includes 98 real images. The resulting model produces highly photorealistic images that accurately follow fine‑grained editing instructions, and a distilled version, LLaDA‑Image‑Turbo, enables fast inference in 2–4 sampling steps. On Qwen‑Image‑Bench, LLaDA‑Image sets new state‑of‑the‑art scores for open‑source models in both English and Chinese tracks, and the authors release weights, code, and detailed recipes to support further research.
arXiv:2607. 06631v1 Announce Type: cross Abstract: Video Diffusion Models (VDMs) have demonstrated superior generation quality but suffer from prohibitive computational costs.
arXiv:2606. 20076v1 Announce Type: cross Abstract: Latent Diffusion Models (LDMs) have become dominant in visual synthesis, but their quality-compute trade-off is largely constrained by the tokenizer's fixed compression ratio.
arXiv:2606. 07098v1 Announce Type: cross Abstract: We present SigmaScale, a method for learning auxiliary scaling matrices $S$ to aid truncated Singular Value Decomposition (SVD) based Large Language Model (LLM) compression.
The paper introduces Abra, a family of flow‑matching transformers used to systematically study scaling laws for text‑to‑image diffusion models across three orders of magnitude in compute. It finds that diffusion models scale predictably like language models but need far more data, with compute optimality occurring at roughly 200 image tokens per parameter—ten times the optimal ratio for large language models. The study also shows that diffusion models are robust to overtraining, that more data is preferable to larger models, and that scaling predictability extends to generative quality, optimal CFG settings, representation quality, and training curve shapes.