arXiv AI By Rahul Sajnani, Yulia Gryaditskaya, Radom\'ir M\v{e}ch, Srinath Sridhar, Matheus Gadelha

Appearance Pointers -- Multimodal Region Control of Diffusion Transformers

Read the original on arXiv AI →

arXiv:2607. 19344v1 Announce Type: cross Abstract: Controllable image generation remains challenging for creative professionals, who often require precise regional control over materials, object identities, and spatial arrangements that cannot be reliably achieved through text prompting alone.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Sep 11

Overpainting: Localized Context-aware Diffusion Image Editing

The paper introduces "overpainting," a localized, context-aware image editing technique that allows users to specify precise or loose editing regions via a trimap. The method adapts a pretrained diffusion model with joint attention and low‑rank adaptation, incorporating attention‑dropout to balance noise, source, and mask inputs. An automated pipeline generates training data by pairing images from language‑based editing models, curating them, and extracting trimaps, enabling the model to perform a wide range of editing tasks.

By Sam Sartor, Iliyan Georgiev, Michael Fischer, Valentin Deschaintre, Pieter Peers
arXiv Computer Vision
Sep 7

RefDiT: Local Attribute Guidance in Reference-Based Image Generation

RefDiT is a new framework for reference-guided image generation that addresses the shortcomings of previous methods when handling complex scenes with multiple objects. It introduces local region guidance by decomposing a single identifier token into attribute-level signals, allowing the model to learn correspondences between tokens and specific regions of a reference image. The approach incorporates a low-rank adapter (LoRA) within a diffusion transformer (DiT) to adjust the inference prompt based on user-provided guidance context, thereby enabling more precise local attribute control.

By Rameshwar Mishra, Srikrishna Karanam, A V Subramanyam