arXiv Computer Vision
Diffusable Latents from Structure-Agnostic Distillation
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Sep 11
CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation
CLIP-RD introduces a relational distillation framework for efficient CLIP knowledge distillation, featuring Vertical Relational Distillation (VRD) and Cross Relational Distillation (XRD). VRD aligns intra‑modal similarity distributions between teacher and student, while XRD aligns cross‑modal similarity distributions to enforce bidirectional symmetry. This joint modeling of multidirectional relational structures improves the student’s embedding geometry, yielding a 1.8%p performance gain over CLIP‑KD across various architectures, tasks, and corruption settings with minimal training‑time overhead.
arXiv AI
Sep 15
Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation
arXiv:2606.03746v3 Announce Type: replace-cross Abstract: Few-step distillation has emerged as a critical component in the development of advanced visual generative foundation models, substantially r...
arXiv Computer Vision
Sep 1
Identity-Conditioned Latent Consistency Distillation for Face Synthesis
arXiv:2608.31053v1 Announce Type: new Abstract: Diffusion models have achieved strong results in high-fidelity image synthesis, but their iterative sampling process makes large-scale generation compu...
Hugging Face Trending Papers
Aug 4
Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging
On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules.
arXiv Machine Learning
Aug 5
Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging
arXiv:2608. 03316v1 Announce Type: new Abstract: On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid.
arXiv Computer Vision
Aug 26
IDeaL: Data-Free Multi-Teacher Distillation via Improved Dead Leaves
The paper introduces IDeaL, a data‑free multi‑teacher distillation technique that generates teacher‑specific, improved samples using decorrelation losses at patch and image levels. By tailoring noise to each teacher, IDeaL produces strong student models that capture complementary teacher information and achieve results close to those distilled from real images. Experiments demonstrate that with only 1,000 images, students trained on IDeaL samples match or exceed the performance of students distilled from a 1,000‑image subset of ImageNet.