arXiv AI

PASTA: Prompt-Aware Annealed Diffusion Sampling

PASTA (Prompt-Aware Shared-Latent Training-Free Annealed Diffusion Sampling) is a new framework for diffusion-based image generation that preserves clean, reserved regions such as headlines, logos, or UI elements. It enforces these constraints by selectively applying semantic guidance only to object-allowed areas during sampling, using a shared latent trajectory and a mask-guided fusion of full-prompt and context-prompt guidance. An annealed schedule suppresses semantic leakage early in denoising and gradually restores visual consistency, leading to improved reserved-region fidelity while maintaining overall image realism and perceptual consistency.

arXiv Computer Vision
Sep 11

Overpainting: Localized Context-aware Diffusion Image Editing

The paper introduces "overpainting," a localized, context-aware image editing technique that allows users to specify precise or loose editing regions via a trimap. The method adapts a pretrained diffusion model with joint attention and low‑rank adaptation, incorporating attention‑dropout to balance noise, source, and mask inputs. An automated pipeline generates training data by pairing images from language‑based editing models, curating them, and extracting trimaps, enabling the model to perform a wide range of editing tasks.

By Sam Sartor, Iliyan Georgiev, Michael Fischer, Valentin Deschaintre, Pieter Peers
arXiv Computer Vision
Sep 3

Diversifying Long Prompt Image Generation through Structured Prompt Embedding Space Sampling

The paper investigates how long, richly detailed prompts cause modern text-to-image models to lose diversity, even when many visual aspects are unspecified. It introduces PromptMoG, a training‑free method that samples prompt embeddings from a Mixture‑of‑Gaussians distribution to restore diversity while preserving semantic fidelity. The authors also present LPD‑Bench, a benchmark for evaluating fidelity and diversity under long, semantically dense prompts, and demonstrate PromptMoG’s effectiveness on four large diffusion models.

By Bo-Kai Ruan, Teng-Fang Hsiao, Ling Lo, Yi-Lun Wu, Hong-Han Shuai
arXiv Machine Learning
Jun 9

Mitigating Diffusion Model Hallucinations with Dynamic Guidance

arXiv:2510. 05356v2 Announce Type: replace-cross Abstract: Hallucinations in diffusion models are samples with structural inconsistencies that can emerge due to the excessive smoothing of the learned score function, which in turn leads to interpolations between modes of the data distribution.

By Kostas Triaridis, Alexandros Graikos, Aggelina Chatziagapi, Grigorios G. Chrysos, Dimitris Samaras
arXiv AI
Jul 22

Appearance Pointers -- Multimodal Region Control of Diffusion Transformers

arXiv:2607. 19344v1 Announce Type: cross Abstract: Controllable image generation remains challenging for creative professionals, who often require precise regional control over materials, object identities, and spatial arrangements that cannot be reliably achieved through text prompting alone.

By Rahul Sajnani, Yulia Gryaditskaya, Radom\'ir M\v{e}ch, Srinath Sridhar, Matheus Gadelha
arXiv AI
Oct 15

QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing

QK-Edit introduces a training‑free query‑key manipulation strategy for Multimodal Diffusion Transformers (MM‑DiTs), enabling precise text‑guided image and video editing. By isolating and adjusting critical attention components, it balances prompt fidelity with structural consistency, supporting tasks such as object addition, removal, replacement, background changes, material and color adjustments, and style transformation. The method works seamlessly with state‑of‑the‑art models like FLUX and HunyuanVideo, improving editing performance without requiring additional training.

By Tiancheng Shen, Zilong Huang, Xiangtai Li, Zhijie Lin, Jiyang Liu, Yitong Wang, Jiashi Feng, Ming-Hsuan Yang, Jun Hao Liew
arXiv Computer Vision
Oct 16

Not Just a Subject: Capturing Adaptive Scene-Specific Variance for Visual Storytelling

EOS-Story is a training‑free framework that improves visual storytelling by balancing identity consistency with scene‑specific adaptation. It separates narrative prompts into identity and scene sub‑prompts and introduces Cross‑Contextual Bridging to inject appearance cues into the scene embedding. In the vision space, Adaptive Identity Preservation aligns identity features to the scene scale and uses Identity‑Aware Self‑Attention to avoid structural artifacts, achieving competitive results on the extended Consistory+ benchmark.

By SeungJu Cha, Ye-Chan Kim, Kwanyoung Lee, Dong-Jin Kim