arXiv Machine Learning

Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models

The paper introduces a two‑stage method for uncovering hidden concepts encoded in Low‑Rank Adaptation (LoRA) adapters used to personalize diffusion models. First, Subspace‑Guided Evolutionary Search (SGES) identifies promising prompts by evaluating their alignment with the LoRA subspace without rendering images. Second, Paired Visual Identification (PVI) renders only the shortlisted prompts and compares LoRA outputs to the base model to confirm coherent, distinctive generation, thereby efficiently revealing undisclosed concepts.

arXiv Machine Learning
Oct 8

ORCA: Hunting Compositional Failures in Text-to-Image Diffusion

ORCA (Orthogonal Residual Compositional Alignment) is a method that improves text-to-image diffusion models by aligning the diffusion transformer’s latent space with a low‑rank target derived from a frozen visual encoder. It introduces an auxiliary loss that uses a predictor with an orthogonal basis parameterised by a learned residual between T5 and CLIP embeddings, providing a prompt‑dependent signal for selecting the visual readout subspace. Experiments on three diffusion‑transformer backbones show that ORCA improves FID and GenEval scores, especially on attribute binding, spatial relations, and multi‑object prompts, without adding inference‑time cost.

By Arshia Hemmat, Amirhossein Vahidi, Amitis Shidani, Mohammad Vali Sanian, Hesam Asadollahzadeh, Aryan Yazdan Parast, Mohammad Lotfollahi
arXiv AI
Jul 8

Few Channels Draw The Whole Picture: Revealing Massive Activations in Diffusion Transformers

arXiv:2605. 13974v2 Announce Type: replace-cross Abstract: Diffusion Transformers (DiTs) and related flow-based architectures are now among the strongest text-to-image generators, yet the internal mechanisms through which prompts shape image semantics remain poorly understood.

By Evelyn Turri, Davide Bucciarelli, Sara Sarto, Lorenzo Baraldi, Marcella Cornia
Hugging Face Trending Papers
Jul 6

LILAC: Layer-Wise Independent LoRAs and Cascaded Conditioning for Multi-Concept Customization of Diffusion Models

Personalizing text-to-image diffusion models to render several specific subjects in a coherent image remains challenging: the model must preserve each subject's identity while keeping the scene spatially and visually coherent. Methods that fuse independently trained concept adapters in a shared weight space (via federated averaging, gradient fusion, or orthogonality constraints) suffer from identity confusion and style bleeding and require joint retraining.

arXiv Machine Learning
Oct 9

C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models

C-LoRA introduces a continual learning approach for pre‑trained visual models that uses a single shared Low‑Rank Adaptation (LoRA) adapter. It employs a learnable routing matrix split into a stability component (R_base) and a plasticity component (R_delta) to balance preserving past knowledge and adapting to new tasks. This design eliminates the need for expanding modules or complex inference fusion while maintaining competitive performance on multiple benchmarks.

By Xin Zhang, Liang Bai, Xian Yang