arXiv Computation and Language
MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Aug 11
Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards
arXiv:2608. 07535v1 Announce Type: cross Abstract: Multi-modal large language models (MLLMs) integrate heterogeneous modalities through modality alignment and fusion, enabling stronger understanding and reasoning.
arXiv AI
Aug 24
ReFrame: Evidence-Guided Test-Time Safety Alignment in Multimodal Large Language Models
ReFrame is a training‑free framework that enhances safety alignment for multimodal large language models at test time. It uses two lightweight agents: one generates risk and utility evidence, and the other rewrites prompts and routes images to create a safe proxy before invoking the deployed MLLM. Experiments show that ReFrame improves jailbreak defense, safety awareness, and reduces over‑sensitivity while maintaining multimodal utility.
arXiv AI
Jun 6
MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
arXiv:2606. 05177v1 Announce Type: cross Abstract: Existing multimodal safety benchmarks focus solely on visual inputs and cannot assess Omni Large Language Models (LLMs) that process vision, audio, and text.
arXiv AI
Sep 3
Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Models
The paper investigates cross‑modal safety drift in multimodal large language models, where a harmless text query paired with a visual image can trigger harmful responses. Empirical analysis identifies unsafe response patterns and shows that visual cues receive limited attention, weakening refusal mechanisms. The authors introduce Safety‑Awareness Representation Transfer (SRT), a lightweight method that transfers safety signals from text processing to mitigate cross‑modal drift while maintaining model utility.
arXiv Machine Learning
Sep 1
MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models
arXiv:2603.02482v2 Announce Type: replace Abstract: Safety evaluation of multimodal large language models requires tracking not only whether an attack succeeds, but also how the interaction unfolds a...
arXiv AI
Aug 19
Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models
The paper introduces Uni-SafeBench, a safety benchmark designed to evaluate the holistic safety of Unified Multimodal Large Models (UMLMs) across six safety categories and seven task types. It also presents Uni-Judger, a framework that separates contextual safety from intrinsic safety to enable rigorous assessment. Evaluations reveal that current unified models do not consistently maintain the safety alignment of their underlying language models, and open‑source UMLMs perform significantly worse on safety than specialized multimodal models, especially in generation tasks.