arXiv AI

Disentangling Dual Image References in Frequency Aware Diffusion Models for Personalized Generation

arXiv AI
Oct 1

ReGain: Restoring Subject Fidelity in Personalization on Synthetic Images

ReGain is a training‑free correction that improves subject fidelity in text‑to‑image diffusion models personalized with synthetic images. The authors show that fine‑tuning on synthetic images degrades fidelity due to inflated classifier‑free guidance, especially at high frequencies. ReGain measures this inflation per frequency band and scales it down during sampling, closing 51‑64% of the fidelity gap on Stable Diffusion v1.5 and improving performance on SDXL and SD 3.5 while preserving text alignment.

By Shubhang Bhatnagar, Ishan Bhatnagar, Viraj Shah, Narendra Ahuja
arXiv Machine Learning
Sep 15

Abstract-LoRA: Unlocking Single-Image Style Transfer through Targeted U-Net Block Training

Abstract‑LoRA introduces a lightweight LoRA training approach that targets specific U‑Net blocks in diffusion models to improve single‑image style transfer. By refining block selection, adding more blocks, and using clustering‑based style abstraction, it better disentangles and balances style and content compared to prior methods like B‑LoRA. Experiments show that the method produces more harmonious artistic images while quantitatively preserving both style and content.

By Xinglin Hu
arXiv AI
Sep 10

MAST: Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfer

MAST (Mask‑Guided Attention Control for Training‑Free Regional‑Multi Style Transfer) is a framework that enables diffusion models to apply multiple reference styles to user‑specified regions of a content image without any training or optimization. It introduces logit‑level attention mass allocation, sharpness‑aware temperature scaling, and discrepancy‑aware detail injection to address mass allocation, selectivity, and detail loss problems in regional‑multi style transfer. Experiments with two to five styles show that MAST outperforms baselines in ArtFID, FID, and R‑FID, achieving high regional style fidelity, content preservation, and scalability.

By Dongkyung Kang, Jaeyeon Hwang, Junseo Park, Minji Kang, Yeryeong Lee, Beomseok Ko, Hanyoung Roh, Jeongmin Shin, Hyeryung Jang
arXiv AI
Sep 3

ContextAnyone: Context-Aware Diffusion for Character-Consistent Text-to-Video Generation

ContextAnyone is a context‑aware diffusion framework that treats a reference image as an explicitly preserved appearance anchor rather than a simple conditioning signal. By jointly reconstructing the reference image and generating the target video within a shared diffusion transformer, it provides direct supervision for maintaining identity and fine‑grained appearance throughout denoising. The method introduces asymmetric information flow and Gap‑RoPE positional representations to keep the reference stable while allowing selective access by video tokens, and demonstrates improved identity and appearance consistency on an OpenVid‑HD benchmark.

By Ziyang Mai, Yu-Wing Tai
arXiv Computer Vision
Sep 7

RefDiT: Local Attribute Guidance in Reference-Based Image Generation

RefDiT is a new framework for reference-guided image generation that addresses the shortcomings of previous methods when handling complex scenes with multiple objects. It introduces local region guidance by decomposing a single identifier token into attribute-level signals, allowing the model to learn correspondences between tokens and specific regions of a reference image. The approach incorporates a low-rank adapter (LoRA) within a diffusion transformer (DiT) to adjust the inference prompt based on user-provided guidance context, thereby enabling more precise local attribute control.

By Rameshwar Mishra, Srikrishna Karanam, A V Subramanyam
Hugging Face Trending Papers
Aug 12

Through Van Gogh's Eyes: Global Style Transfer with Diffusion Mod

Artistic image synthesis aims to recreate the expressive visual identity of a target artist, yet existing methods often fail to capture an artist's global style. Conventional style transfer methods transfer the style of one or a few reference artworks to a content image in a One-to-One manner, making them effective for artwork-level stylization but limited in representing the broader stylistic distribution of an artist.