Personalizing text-to-image diffusion models to render several specific subjects in a coherent image remains challenging: the model must preserve each subject's identity while keeping the scene spatially and visually coherent. Methods that fuse independently trained concept adapters in a shared weight space (via federated averaging, gradient fusion, or orthogonality constraints) suffer from identity confusion and style bleeding and require joint retraining.
arXiv:2510.23515v3 Announce Type: replace
Abstract: This paper proposes FreeFuse, a training-free framework for multi-subject text-to-image generation through automatic fusion of multiple subject LoR...
By Yaoli Liu, Yao-Xiang Ding, Kun Zhou
arXiv:2609.37198v1 Announce Type: new
Abstract: Pretrained text-to-image models contain broad visual knowledge, yet they cannot reliably acquire or refine a specific visual identity from only a few r...
By Haoran He, Runyuan Cai, Yiming Wang, Lin Yu, Xiaodong Zeng
arXiv:2609.01433v1 Announce Type: new
Abstract: Concept erasure aims to suppress unsafe, privacy-sensitive, or undesirable generations in text-to-image diffusion models while preserving benign semant...
By Qinghui Gong, Xunlei Chen, Yu-Xuan Zhang, Hua Meng, Zhengchun Zhou
arXiv:2608. 14172v1 Announce Type: cross Abstract: Text-to-image diffusion models have two major drawbacks that severely limit their practical utility: (1) standard models lack an intrinsic mechanism for continuous, concept-specific guidance (e.
By Nikolai R\"ohrich, Isabell Hans, Felix Krause, Bj\"orn Ommer
Low-Rank Adaptation (LoRA) has made diffusion model personalization affordable and widely shareable, but it also creates moderation risks when harmful or problematic content is not faithfully disclose...
The paper introduces a two‑stage method for uncovering hidden concepts encoded in Low‑Rank Adaptation (LoRA) adapters used to personalize diffusion models. First, Subspace‑Guided Evolutionary Search (SGES) identifies promising prompts by evaluating their alignment with the LoRA subspace without rendering images. Second, Paired Visual Identification (PVI) renders only the shortlisted prompts and compares LoRA outputs to the base model to confirm coherent, distinctive generation, thereby efficiently revealing undisclosed concepts.
By Yupeng Qiu, Han Fang, Dhananjai Sharma, Ee-Chien Chang
SeqLoRA introduces a bilevel optimization framework for continual learning of text‑to‑image diffusion models, jointly optimizing LoRA factors while enforcing orthogonality between new and existing concept bases. The method achieves monotone descent, convergence guarantees, and a high‑probability bound on catastrophic forgetting that depends on residual interference energy. Experiments on Stable Diffusion with up to 101 concepts demonstrate improved identity preservation, minimal cross‑concept leakage, no need for a fusion step, and scalability beyond memory limits of fusion‑based approaches.
By Javad Parsa, Enis Simsar, Amir Joudaki, Thomas Hofmann, Andr\'e M. H. Teixeira
ReGain is a training‑free correction that improves subject fidelity in text‑to‑image diffusion models personalized with synthetic images. The authors show that fine‑tuning on synthetic images degrades fidelity due to inflated classifier‑free guidance, especially at high frequencies. ReGain measures this inflation per frequency band and scales it down during sampling, closing 51‑64% of the fidelity gap on Stable Diffusion v1.5 and improving performance on SDXL and SD 3.5 while preserving text alignment.
By Shubhang Bhatnagar, Ishan Bhatnagar, Viraj Shah, Narendra Ahuja
arXiv:2606. 03792v1 Announce Type: cross Abstract: Low-Rank Adaptation (LoRA) successfully enables personalization in text-to-image generation by adapting pre-trained diffusion models to specific visual concepts and styles.
By Georgios Tsoumplekas, Stella Bounareli, Vasileios Argyriou
arXiv:2610.28567v1 Announce Type: cross
Abstract: Text-to-image personalization aims to equip pretrained diffusion models with the ability to learn user-specific concepts from only a few reference im...
By Yuzhi Liu, Enis Simsar, Giulia Lanzillotta, Thomas Hofmann
ORCA (Orthogonal Residual Compositional Alignment) is a method that improves text-to-image diffusion models by aligning the diffusion transformer’s latent space with a low‑rank target derived from a frozen visual encoder. It introduces an auxiliary loss that uses a predictor with an orthogonal basis parameterised by a learned residual between T5 and CLIP embeddings, providing a prompt‑dependent signal for selecting the visual readout subspace. Experiments on three diffusion‑transformer backbones show that ORCA improves FID and GenEval scores, especially on attribute binding, spatial relations, and multi‑object prompts, without adding inference‑time cost.
By Arshia Hemmat, Amirhossein Vahidi, Amitis Shidani, Mohammad Vali Sanian, Hesam Asadollahzadeh, Aryan Yazdan Parast, Mohammad Lotfollahi