arXiv AI

MAST: Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfer

MAST (Mask‑Guided Attention Control for Training‑Free Regional‑Multi Style Transfer) is a framework that enables diffusion models to apply multiple reference styles to user‑specified regions of a content image without any training or optimization. It introduces logit‑level attention mass allocation, sharpness‑aware temperature scaling, and discrepancy‑aware detail injection to address mass allocation, selectivity, and detail loss problems in regional‑multi style transfer. Experiments with two to five styles show that MAST outperforms baselines in ArtFID, FID, and R‑FID, achieving high regional style fidelity, content preservation, and scalability.

arXiv Computer Vision
Sep 11

Overpainting: Localized Context-aware Diffusion Image Editing

The paper introduces "overpainting," a localized, context-aware image editing technique that allows users to specify precise or loose editing regions via a trimap. The method adapts a pretrained diffusion model with joint attention and low‑rank adaptation, incorporating attention‑dropout to balance noise, source, and mask inputs. An automated pipeline generates training data by pairing images from language‑based editing models, curating them, and extracting trimaps, enabling the model to perform a wide range of editing tasks.

By Sam Sartor, Iliyan Georgiev, Michael Fischer, Valentin Deschaintre, Pieter Peers
arXiv Machine Learning
Sep 15

Abstract-LoRA: Unlocking Single-Image Style Transfer through Targeted U-Net Block Training

Abstract‑LoRA introduces a lightweight LoRA training approach that targets specific U‑Net blocks in diffusion models to improve single‑image style transfer. By refining block selection, adding more blocks, and using clustering‑based style abstraction, it better disentangles and balances style and content compared to prior methods like B‑LoRA. Experiments show that the method produces more harmonious artistic images while quantitatively preserving both style and content.

By Xinglin Hu
arXiv AI
Oct 15

QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing

QK-Edit introduces a training‑free query‑key manipulation strategy for Multimodal Diffusion Transformers (MM‑DiTs), enabling precise text‑guided image and video editing. By isolating and adjusting critical attention components, it balances prompt fidelity with structural consistency, supporting tasks such as object addition, removal, replacement, background changes, material and color adjustments, and style transformation. The method works seamlessly with state‑of‑the‑art models like FLUX and HunyuanVideo, improving editing performance without requiring additional training.

By Tiancheng Shen, Zilong Huang, Xiangtai Li, Zhijie Lin, Jiyang Liu, Yitong Wang, Jiashi Feng, Ming-Hsuan Yang, Jun Hao Liew
arXiv AI
Oct 26

PASTA: Prompt-Aware Annealed Diffusion Sampling

PASTA (Prompt-Aware Shared-Latent Training-Free Annealed Diffusion Sampling) is a new framework for diffusion-based image generation that preserves clean, reserved regions such as headlines, logos, or UI elements. It enforces these constraints by selectively applying semantic guidance only to object-allowed areas during sampling, using a shared latent trajectory and a mask-guided fusion of full-prompt and context-prompt guidance. An annealed schedule suppresses semantic leakage early in denoising and gradually restores visual consistency, leading to improved reserved-region fidelity while maintaining overall image realism and perceptual consistency.

By Ren Ajiki, Ryugo Morita, Stanislav Frolov, Brian Bernhard Moser, Ko Watanabe, Riku Takahashi, Andreas Dengel, Jinjia Zhou
arXiv Computer Vision
Sep 21

SafeStyle: Calibrated Style Residual Injection for Controllable Style-Leakage Trade-off in Diffusion Stylization

SafeStyle is a training‑free framework that injects calibrated style residuals into frozen diffusion models for reference‑guided stylization. It estimates style‑supported and content‑associated subspaces from small calibration sets, then transports purified style evidence across adaptive spatial granularity while limiting its influence with a residual‑norm budget. Experiments on texture‑ and geometry‑dominant styles show high DINO style similarity (0.432–0.474) with minimal semantic leakage (0.8%).

By Zhangping Yang, Min Li, Song Yan, Rong Gao, Xinliang Bi, Guanye Xiong, Yujie He
arXiv AI
5d ago

LociArt: A Unified Paradigm for Free-Style Local Video Stylization

LociArt introduces a unified framework for free‑style local video stylization, allowing precise specification of where to stylize and which style to apply while preserving non‑target content. The core method, GRNStyle, builds on Generative Refinement Networks and uses a polar style‑aware mask to embed style identities as equal‑norm directions within selected regions. An automated pipeline generates paired supervision, leading to the LociArt‑100K dataset and benchmark, which evaluate regional style quality, non‑target preservation, and temporal consistency, and experiments show GRNStyle outperforms existing methods in these aspects.

By Yuxuan Chen, Feng Xie, Jiagao Hu, Fuhao Li, Zepeng Wang, Fei Wang, Daiguo Zhou