Hugging Face Blog

Faster Stable Diffusion with Core ML on iPhone, iPad, and Mac

arXiv Machine Learning
Sep 11

Understanding LoRA Rank Trade-offs in Diffusion Model Fine-Tuning

The study investigates how the rank of Low‑Rank Adaptation (LoRA) affects diffusion model fine‑tuning on CIFAR‑10 using a DDPM U‑Net. Experiments with ranks 2, 4, 8, 16, and 32 show that moderate ranks—particularly rank 4—yield the best FID scores while keeping trainable parameters, runtime, and GPU memory low. Higher ranks offer only marginal improvements at a higher computational cost, suggesting that small‑to‑moderate ranks are efficient defaults for fixed training budgets.

By Iman Khazrak, Narges Nejad, Mostafa M. Rezaee, Robert C. Green II