arXiv Computer Vision By Chenyu Zhang, Lanjun Wang, Yueyang Cheng, Ruidong Chen, Wenhui Li, An-an Liu

What Concepts Lie Within? Detecting and Suppressing Risky Content in Diffusion Transformers

Read the original on arXiv Computer Vision →

The paper introduces a training‑free, inference‑time safeguard for Diffusion‑Transformer (DiT) text‑to‑image models that detects and suppresses risky content such as sexual, violent, or copyrighted imagery. By analyzing attention heads, the authors find that each head is sensitive to specific semantic concepts, enabling the creation of an Attention Head Vector (AHV) that serves as a discriminative signature for risky tokens. During generation, a momentum‑based strategy tracks token‑wise AHVs across denoising steps, and a sensitivity‑guided suppression mechanism dynamically reduces attention logits for identified risky tokens, effectively mitigating harmful outputs while maintaining visual quality and robustness against adversarial prompts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Machine Learning
Sep 14

Certifying Concept Unlearning in Text-to-Image Diffusion Models

The paper introduces a certification framework for assessing concept unlearning in text-to-image diffusion models, offering high‑confidence guarantees with bounded error on residual concept leakage. Unlike prior methods that rely solely on attack success rates from automated prompt searches, this approach combines statistical certification with worst‑case analysis along concept‑relevant embedding directions to derive explicit upper bounds on leakage probability. Evaluations across NSFW content, artistic styles, and celebrity identities reveal that certified leakage bounds exceed standard attack success rates by 16.2%, highlighting significant residual risks overlooked by existing protocols.

By Mansi, Luca Marzari, Francesco Leofante
arXiv AI
Sep 15

DSS: Dynamic Semantic Steering for Robust Concept Erasure in Diffusion Models

The paper introduces Dynamic Semantic Steering (DSS), a training‑free, inference‑time defense for robust concept erasure in text‑to‑image diffusion models. DSS models local semantic neighborhoods geometrically, automatically identifies benign semantic anchors, and applies context‑aware, constrained feature correction using cross‑attention signals. Experiments show DSS achieves an average erasure rate of 91.0%, outperforming prior defenses while reducing semantic drift and preserving generation fidelity.

By Qinghui Gong, Zhengchun Zhou, Hua Meng, Yihuai Liang, Yuxuan Zhang
arXiv Computer Vision
Nov 5

NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers

NitroFusion2 introduces Text-Stream Adversarial Distillation (TSAD) and Aspect-Cued Discrimination (ACD) to use intermediate text-token states as adversarial representations in multimodal diffusion transformers. By conditioning the discriminator on aspects like typography, objects, or style, the method steers which properties are emphasized during training. The resulting model achieves comparable quality to conventional 4–8 step distillation in just 1–2 steps, producing sharp text and fine-grained details in one-step generation.

By Dar-Yen Chen, Yi-Zhe Song
Hugging Face Trending Papers
Jul 7

AEGIS: A Mechanism-Guided Defense against Visual Synonym Jailbreaks in Text-to-Image Models

Text-to-image diffusion models have achieved high visual fidelity and broad adoption, but remain vulnerable to safety violations when adversaries exploit them to synthesize illicit content. Existing alignment paradigms, from input sanitization to structural feature pruning, are largely organized around unsafe concepts explicitly exposed during filtering, editing, or localization.