arXiv AI August 28, 2026 By Kaichao Jiang, Changtao Miao, Baiqi Wu, Zhiyuan Lu, Kang Yang, Peiwei Zhao, Junchi Chen, Yunfeng Diao, He Liu, Qi Chu, Tao Gong, Nenghai Yu

Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation

Read the original on arXiv AI →

Multi2AV‑Safety is a new benchmark for evaluating safety in multimodal-to-audio‑video generation. It covers all 11 non‑singleton conditioning configurations (text, image, audio, video) and contains 11,024 attack instances. The benchmark reveals that safety guards often fail when harmful semantics arise from combinations of benign inputs or when explicit harmful cues are masked by benign multimodal context, highlighting a gap in compositional risk perception.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Sep 10

AV-SafetyBench: A Safety Benchmark for Text-to-Audio-Video Generation

arXiv:2609.06991v1 Announce Type: cross Abstract: Recent text-to-audio-video (T2AV) models jointly generate video, speech, sound effects, and ambience from a single text prompt. This capability poses...

By Suah Choi, Tae-Young Lee, Gyeong-Moon Park

diffusion multimodal benchmarks safety

arXiv Computation and Language

Sep 21

MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs

arXiv:2609.20850v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) show remarkable advancements, their cross-modal capabilities introduce complex vulnerabilities that easi...

By Yueming Lyu, Yilian Shi, Haoxiang Tan, Linzhuang Zou, Qihao Wang, Guihua Yu, Jie Qin, Xin Gao, Chenyang Si, Jing Dong, Caifeng Shan

llms multimodal benchmarks safety

arXiv Computation and Language

Aug 25

Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models

Omni‑SafetyBench is a new benchmark designed to evaluate the safety of Omni‑Modal Large Language Models (OLLMs) that process visual, auditory, and textual data. It contains 23,328 test instances across 24 modality variations derived from 972 seed samples, and introduces metrics such as Safety‑score (based on Conditional Attack Success Rate and Conditional Refusal Rate) and Cross‑Modal Safety Consistency score. Evaluation of 11 state‑of‑the‑art OLLMs shows severe vulnerabilities, with only three models achieving a Safety‑score above 0.6 and safety degrading sharply for audio‑visual inputs, underscoring the need for improved safety alignment methods.

By Leyi Pan, Zheyu Fu, Yunpeng Zhai, Shuchang Tao, Sheng Guan, Shiyu Huang, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Felix Henry, Aiwei Liu, Lijie Wen

llms multimodal benchmarks safety

arXiv Machine Learning

Sep 1

MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models

arXiv:2603.02482v2 Announce Type: replace Abstract: Safety evaluation of multimodal large language models requires tracking not only whether an attack succeeds, but also how the interaction unfolds a...

By Zhongxi Wang, Yueqian Lin, Jingyang Zhang, Zinuo Cheng, Hai Helen Li, Yiran Chen

llms multimodal safety

arXiv Computer Vision

Sep 16

Unsafe by Reciprocity: How Generation-Understanding Coupling Undermines Safety in Unified Multimodal Models

The paper examines how the tight integration of multimodal understanding and image generation in Unified Multimodal Models (UMMs) can create new safety vulnerabilities. It introduces RICE, an attack framework that exploits bidirectional interactions between generation and understanding to propagate unsafe signals across modalities. Experiments demonstrate high attack success rates in both Generation‑to‑Understanding and Understanding‑to‑Generation pathways, revealing previously overlooked safety weaknesses in UMMs.

By Kaishen Wang, Heng Huang

llms diffusion multimodal safety

arXiv AI

Jun 2

Jailbreaking Multimodal Large Language Models using Multi-Clip Video

arXiv:2606. 02111v1 Announce Type: cross Abstract: As multimodal large language models (MLLMs) have advanced to process video inputs, concerns have emerged about their potential for malicious misuse.

By Choongwon Kang, Seungjong Sun, Hyunmin Jun, Jang Hyun Kim