Rethinking Memorization Mitigation in Diffusion Models: Reinforcing Text Conditioning
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
arXiv:2603. 00133v2 Announce Type: replace-cross Abstract: Generative models have been shown to "memorize" certain training data, leading to verbatim or near-verbatim generating images, which may cause privacy concerns or copyright infringement.
The paper introduces Reachability-Aware Diffusion Steering (RADS), an offline-trained, sampling-time control framework designed to reduce memorization in text-to-image diffusion models while preserving image quality and prompt alignment. RADS treats the diffusion denoising process as a dynamical system and learns a policy-conditioned reachability value function that predicts future memorization risk along the denoising trajectory. By formulating a constrained reinforcement learning problem, the policy steers the trajectory away from memorization through perturbations in the caption embedding space, achieving a better trade‑off between generation diversity, quality, and alignment than existing inference‑time baselines.
arXiv:2606. 14125v1 Announce Type: cross Abstract: Inversion-based image editing offers flexible and training-free control but still struggles with inversion accuracy and the trade-off between editing fidelity and background preservation.
PASTA (Prompt-Aware Shared-Latent Training-Free Annealed Diffusion Sampling) is a new framework for diffusion-based image generation that preserves clean, reserved regions such as headlines, logos, or UI elements. It enforces these constraints by selectively applying semantic guidance only to object-allowed areas during sampling, using a shared latent trajectory and a mask-guided fusion of full-prompt and context-prompt guidance. An annealed schedule suppresses semantic leakage early in denoising and gradually restores visual consistency, leading to improved reserved-region fidelity while maintaining overall image realism and perceptual consistency.
The paper introduces StruPO, a structure‑aware prompt editor that uses a calibrated scene graph to guide prompt modifications while preserving requested scene attributes. By aligning token edits with graph‑based structural feedback, StruPO learns a reusable editing policy that improves faithfulness and relational fidelity across multiple generation backbones. Experiments on diverse test sets and human studies show consistent gains in image quality and preference, especially for complex relations.
Diffusion Language Models (DLMs) have recently achieved substantial progress in natural language generation tasks. Recent research demonstrates that adaptive token generation ordering can significantly improve performance in mathematical reasoning and code synthesis applications.