Continual Concept Personalization in Diffusion Models via Residual-Space Adaptation
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Text-to-image personalization aims to equip pretrained diffusion models with the ability to learn user-specific concepts from only a few reference images. Existing methods, however, typically train ea...
arXiv:2610.01331v1 Announce Type: new Abstract: Continual personalization of text-to-image diffusion models requires sequentially acquiring new concepts while retaining previously learned ones. Howev...
arXiv:2511.22245v2 Announce Type: replace Abstract: Personalizing text-to-image diffusion models extends pretrained models to represent novel user-specific concepts from only a few reference images....
SeqLoRA introduces a bilevel optimization framework for continual learning of text‑to‑image diffusion models, jointly optimizing LoRA factors while enforcing orthogonality between new and existing concept bases. The method achieves monotone descent, convergence guarantees, and a high‑probability bound on catastrophic forgetting that depends on residual interference energy. Experiments on Stable Diffusion with up to 101 concepts demonstrate improved identity preservation, minimal cross‑concept leakage, no need for a fusion step, and scalability beyond memory limits of fusion‑based approaches.
The paper presents a preference-driven unlearning framework for few-step distilled text-to-image diffusion models. It identifies that standard Direct Preference Optimization (DPO) fails on these models due to altered generation dynamics and proposes a modified formulation aligned with few-step properties. The method enables direct concept removal while preserving desirable capabilities, and is evaluated on identity, NSFW, and object-level unlearning tasks with strong forgetting and retention results.
arXiv:2605.19750v2 Announce Type: replace Abstract: Visual autoregressive (VAR) models have recently emerged as an efficient paradigm for text-to-image generation, yet their personalization capabilit...