arXiv Machine Learning

CARE: Condition-Aware Representation Regularization for Diffusion Models

The paper introduces CARE, a lightweight, plug‑and‑play regularization framework for diffusion models that dynamically adjusts feature distributions based on condition similarity. By leveraging built‑in conditioning signals such as labels or text prompts, CARE promotes tighter feature clusters for similar conditions without requiring explicit alignment losses or external supervision. Empirical results show consistent improvements in visual fidelity and convergence stability, achieving significant FID reductions and speed‑ups on ImageNet and text‑to‑image tasks, and it can be combined with existing regularization methods for further gains.

arXiv Computer Vision
Sep 11

AcFlow: Controlling Text-to-Image Diffusion Transformers via Learned Conditional Activation Flow

AcFlow introduces an inference‑time controller for text‑to‑image diffusion transformers that transports intermediate layer activations through a learned, concept‑conditioned velocity field while keeping the base model frozen. The method allows fine‑grained style intensity control and suppression of unwanted concepts, achieving superior style–content trade‑offs compared to baselines and generalizing to unseen concepts without per‑concept fitting. Experiments demonstrate improved style alignment and qualitative suppression of diverse concepts where direct prompting fails.

By Junran Wang, Zehao Jin, Tianyu Luan, Xinjie Shen
arXiv AI
Oct 19

Rethinking the design space of Vanilla Pixel Diffusion Transformers

The paper revisits pixel-space diffusion models, showing that a vanilla diffusion transformer can generate directly in pixel space by swapping velocity prediction for clean image prediction. It introduces MFlow, an adaptive parameterization that interpolates between these two prediction modes based on data representation and model capacity, and demonstrates competitive performance on ImageNet, UCF‑101, and text‑to‑image tasks. Additionally, the authors add ConvRefiner, a lightweight convolutional module trained post‑hoc to restore high‑frequency details while preserving FID.

By Soumik Mukhopadhyay, Long Mai, Abhinav Shrivastava, Aniruddha Mahapatra
arXiv AI
Oct 1

ReGain: Restoring Subject Fidelity in Personalization on Synthetic Images

ReGain is a training‑free correction that improves subject fidelity in text‑to‑image diffusion models personalized with synthetic images. The authors show that fine‑tuning on synthetic images degrades fidelity due to inflated classifier‑free guidance, especially at high frequencies. ReGain measures this inflation per frequency band and scales it down during sampling, closing 51‑64% of the fidelity gap on Stable Diffusion v1.5 and improving performance on SDXL and SD 3.5 while preserving text alignment.

By Shubhang Bhatnagar, Ishan Bhatnagar, Viraj Shah, Narendra Ahuja
Hugging Face Trending Papers
Jun 19

Adversarial Domain Prompt Tuning and Generation for Single Domain Generalization

Single domain generalization (SDG) aims to learn a robust model, which could perform well on many unseen domains while there is only one single domain available for training. One of the promising directions for achieving single-domain generalization is to generate out-of-domain (OOD) training data through data augmentation or image generation.

arXiv Machine Learning
Oct 16

Purify Before You Align: Improving Representation Alignment for Diffusion Models

The paper proposes Local Appearance Purification (LAP), a method that refines the alignment target for diffusion models by removing locally predictable content and focusing on the residual. LAP improves generation quality across multiple pretrained encoders, notably reducing FID from 53.1 to 42.3 on ImageNet-256 with MAE. The authors provide a theoretical framework linking alignment effectiveness to the information emphasized by the target and introduce a constant‑readout test to predict purification benefits.

By Yingheng Wang, Yaoqiang Li, Yaqin Wu, Junwen Bai, Jiatao Gu, Christopher De Sa, Volodymyr Kuleshov
arXiv Computer Vision
Nov 6

OptIMerge: Optimization-Based Merging of Diffusion Models

OptIMerge is an optimization-based framework that merges multiple image‑to‑image diffusion models fine‑tuned for dense, pixel‑accurate tasks. Instead of naive scalar averaging, it learns lightweight, architecture‑aware merger coefficients—such as low‑rank feature alignment for attention layers and per‑kernel coefficients for convolutional layers—by minimizing a multi‑task diffusion loss on a small image set. The approach keeps the expert models frozen, uses less than 0.4% of the network parameters, and achieves multi‑task performance comparable to joint training while outperforming zero‑shot model soups.

By Viraj Shah, Ashwini Pokle, Peyman Milanfar, Mauricio Delbracio, Hossein Talebi
arXiv AI
Jul 8

TILDE: TILt-based Distributional Erasure for Concept Unlearning

arXiv:2607. 06432v1 Announce Type: cross Abstract: Concept unlearning in text-to-image diffusion models is critical for safe and practical deployment: with rising privacy concerns, copyright disputes, trademark constraints, and safety regulations, deployed systems must be able to suppress unwanted concepts after training.

By Naveen George, Naoki Murata, Yuhta Takida, Konda Reddy Mopuri, Yuki Mitsufuji