arXiv Machine Learning

Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models

The paper presents a preference-driven unlearning framework for few-step distilled text-to-image diffusion models. It identifies that standard Direct Preference Optimization (DPO) fails on these models due to altered generation dynamics and proposes a modified formulation aligned with few-step properties. The method enables direct concept removal while preserving desirable capabilities, and is evaluated on identity, NSFW, and object-level unlearning tasks with strong forgetting and retention results.

arXiv AI
Jul 8

TILDE: TILt-based Distributional Erasure for Concept Unlearning

arXiv:2607. 06432v1 Announce Type: cross Abstract: Concept unlearning in text-to-image diffusion models is critical for safe and practical deployment: with rising privacy concerns, copyright disputes, trademark constraints, and safety regulations, deployed systems must be able to suppress unwanted concepts after training.

By Naveen George, Naoki Murata, Yuhta Takida, Konda Reddy Mopuri, Yuki Mitsufuji
arXiv Computer Vision
Oct 2

RASteer: Retain-Aware Activation Steering for Concept Erasure in Diffusion Models

RASteer is a training‑free method for concept erasure in text‑to‑image diffusion models that retains other concepts. It constructs a retain subspace from concepts to preserve, then removes components aligned with this subspace from the erasure direction using Retain‑Orthogonal Steering (ROS). Overlap‑Adaptive Calibration (OAC) further adjusts the removal of shared components at each layer and denoising step, balancing target erasure with concept preservation. Experiments show RASteer matches or outperforms existing activation steering and weight editing baselines on unsafe‑content, instance, and artistic‑style erasure tasks across multiple backbones and benchmarks.

By Yongliang Wu, Haori Lu, Yulun Wu, Jinqi Luo, Xingyu Zhu, Yaoyao Liu
arXiv Computer Vision
Sep 30

Weeding Out Bad Seeds: Initial-Noise-Robust Unlearning for Text-to-Image Diffusion Models

arXiv:2609.37537v1 Announce Type: new Abstract: Machine unlearning has emerged as a critical post-hoc safety measure to erase sensitive concepts from Text-to-Image (T2I) models without prohibitive re...

By Arian Komaei Koma, Seyed Amir Kasaei, Aida Aryafar, Matin Ghiasi, Ali Aghayari, Amirhossein Souri, Mohammad Mosayyebi, AmirMahdi Sadeghzadeh, Mohammad Hossein Rohban
arXiv Machine Learning
Sep 7

Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models

The paper introduces Diffusion LAIR, a listwise preference optimization technique that leverages continuous reward scores instead of binary pairwise comparisons to align text‑to‑image diffusion models. LAIR transforms reward scores into centered advantage weights and optimizes an advantage‑weighted regression objective on an implicit reward defined by denoising‑loss improvement over a reference model, with a quadratic penalty to regulate reward magnitude. Experiments demonstrate that Diffusion LAIR surpasses strong baseline methods on SD1.5 and SDXL across generation, compositional, and editing tasks.

By Austin Wang, Jiaqi Han, Stefano Ermon, Yisong Yue
arXiv Machine Learning
Sep 30

Reference-Guided Machine Unlearning

Reference-Guided Machine Unlearning (ReGUn) is a vision unlearning framework that prioritizes distributional indistinguishability over degradation-based heuristics. It uses disjoint held-out data to create a class-conditioned reference distribution for distillation, guiding forget samples toward non-member behavior without explicitly degrading predictions. Experiments across various architectures and datasets show that ReGUn achieves a competitive forgetting–utility trade-off and closely matches retrain-like membership inference behavior.

By Jonas Mirlach, Sonia Laguna, Julia E. Vogt
arXiv AI
Oct 1

ReGain: Restoring Subject Fidelity in Personalization on Synthetic Images

ReGain is a training‑free correction that improves subject fidelity in text‑to‑image diffusion models personalized with synthetic images. The authors show that fine‑tuning on synthetic images degrades fidelity due to inflated classifier‑free guidance, especially at high frequencies. ReGain measures this inflation per frequency band and scales it down during sampling, closing 51‑64% of the fidelity gap on Stable Diffusion v1.5 and improving performance on SDXL and SD 3.5 while preserving text alignment.

By Shubhang Bhatnagar, Ishan Bhatnagar, Viraj Shah, Narendra Ahuja