arXiv AI

InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation

InGuard introduces an inner guardrail for text-to-image generation that operates within the model’s own representations, avoiding external classifiers. It grades prompts using the text encoder’s embeddings, modifies risky embeddings with SAGE to produce safe images, and employs a latent detector to halt generation early. Evaluated on the RevGen Safety Benchmark, InGuard achieves a 97.9–98.8% safety rate across five open-weight models while reducing benign disturbances, model parameters, and denoising steps.

arXiv AI

Aug 19

DiSCO: Defending text-to-image generation through distribution-guided contrastive prompt optimization

DiSCO is a zero‑shot, black‑box defense for text‑to‑image models that operates solely at the prompt level. It expands prompts with a distribution‑guided suffix using beam search and contrastive scoring against safe and unsafe image pools generated by the target model, iteratively refining until safe content is produced. The method improves safety on the I2P benchmark under various red‑teaming attacks, reducing attack success rates by 37.7% and 25.13% while preserving semantic fidelity and image coherence.

By Tong Zhang, Motasem Alfarra, Carlos Hinojosa, Christos Louizos, Bernard Ghanem

llms diffusion fine-tuning multimodal benchmarks safety

arXiv AI

Jul 28

AI-generated Images Challenge Visual Trust in High-risk Scenarios

arXiv:2607. 22745v1 Announce Type: cross Abstract: Rapid advances in image generation are eroding the evidentiary value of visual content in settings where authenticity can affect public safety and personal reputation.

By Yi-Zhi Wang, Yichen Xiao, Linan Yue, Weibo Gao, Yichao Du, Pengfei Fang, Shimin Di, Min-Ling Zhang

llms diffusion multimodal benchmarks safety

arXiv AI

Aug 19

COMIC: Reference-Aware Safety Gating for Multimodal Large Language Models

The paper introduces COMIC, a reference‑aware safety gate designed for multimodal large language models (MLLMs). COMIC detects the operation requested by a user, identifies visual targets through OCR and open‑vocabulary proposals, and evaluates safety on explicit operation‑target pairs, using max‑risk aggregation and quality‑aware routing to decide whether to allow or block a request. Experiments on several open‑source MLLMs and jailbreak benchmarks show that COMIC improves robustness while maintaining benign utility and efficiency.

By Md Abdullahil Oaphy, Anhao Xiang, Zongxing Xie, Huayue Gu, Chenyu Wang, Honghui Xu

llms computer-vision multimodal benchmarks safety

arXiv AI

Jun 6

Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generation

arXiv:2606. 05290v1 Announce Type: cross Abstract: Recent progress in generative modeling has made safety control a central challenge, yet existing approaches remain largely model-specific, requiring retraining or tailored interventions for each new architecture.

By Tobia Poppi, Silvia Cappelletti, Sara Sarto, Florian Schiffers, Garin Kessler, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

llms diffusion multimodal safety

Hugging Face Trending Papers

Jul 7

AEGIS: A Mechanism-Guided Defense against Visual Synonym Jailbreaks in Text-to-Image Models

Text-to-image diffusion models have achieved high visual fidelity and broad adoption, but remain vulnerable to safety violations when adversaries exploit them to synthesize illicit content. Existing alignment paradigms, from input sanitization to structural feature pruning, are largely organized around unsafe concepts explicitly exposed during filtering, editing, or localization.

diffusion efficiency multimodal safety

arXiv Computer Vision

Aug 21

Reading Between the Pixels: An Inscriptive Jailbreak Attack on Text-to-Image Models

arXiv:2604. 05853v3 Announce Type: replace Abstract: Modern text-to-image (T2I) models can now render legible, paragraph-length text, enabling a fundamentally new class of misuse.

By Zonghao Ying, Haowen Dai, Lianyu Hu, Zonglei Jing, Quanchen Zou, Yaodong Yang, Aishan Liu, Xianglong Liu

llms diffusion robotics multimodal benchmarks safety

arXiv AI

Aug 25

GuardPaint:SpeculativeSafetyDecodingforText-to-ImageGeneration

arXiv:2608.21869v1 Announce Type: cross Abstract: Text-to-image (T2I) diffusion models offer powerful visual generation, but their controllability creates a critical safety challenge: adversarial pro...

By Shreyash Dhoot, Paras Dhiman, Arsh Abbas Naqvi, Aranbi Dutta, Aman Chadha, Vinija Jain, Amitava Das

diffusion computer-vision safety

arXiv AI

Jun 26

Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks

arXiv:2606. 27147v1 Announce Type: cross Abstract: Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens.

By Yunqi Xue, Zhijiang Li, Philip Torr, Jindong Gu

rag diffusion fine-tuning efficiency multimodal safety

arXiv AI

Jul 2

The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models

arXiv:2607. 00402v1 Announce Type: cross Abstract: Safety alignment of text-to-image (T2I) diffusion models aims to suppress harmful generations while preserving utility on benign prompts.

By Adeel Yousaf, Soumik Ghosh, James Beetham, Amrit Singh Bedi, Mubarak Shah

rag diffusion nlp benchmarks safety

arXiv AI

Sep 1

Not Safe for All: Auditing the Dialect Penalty in Text-to-Image Safety Pipelines

arXiv:2608.29589v1 Announce Type: new Abstract: Text-to-image (T2I) safety guardrails fail to generalize equitably to non-standard dialects. Evaluating 23,080 paired prompts across five English diale...

By Minkyu Kim, Juhwan Choi, YoungBin Kim

diffusion benchmarks safety

arXiv Computer Vision

Sep 22

BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models

arXiv:2511.18921v2 Announce Type: replace Abstract: Backdoor attacks undermine the reliability and trustworthiness of machine learning systems by injecting hidden behaviors that can be maliciously ac...

By Juncheng Li, Yige Li, Hanxun Huang, Yunhao Chen, Xin Wang, Yixu Wang, Xingjun Ma, Yu-Gang Jiang

llms nlp multimodal benchmarks safety

arXiv AI

Aug 11

Decoy Images Amplify Caption-Mediated Defenses Against Encoded Jailbreaks

arXiv:2608. 01043v2 Announce Type: replace-cross Abstract: We report a counter-intuitive interaction between image inputs and existing black-box defenses on Vision--Language Models (VLMs): pairing an encoded jailbreak prompt with an unrelated decoy image can sharply lower attack success rate (ASR).

By Haoyu Zhang, Xiangchen Guan, Shibo Zheng, Mohammad Zandsalimy, Shanu Sushmita

llms multimodal safety

Related stories