arXiv AI
Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderation
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 19
Learning to Reason About Implicit Multimodal Harm: From Diagnosis to Generalization
The paper introduces MuPHI, a diagnostic benchmark that isolates multimodal harm arising solely from image‑text composition, providing benign counterfactuals and annotated harm rationales. Evaluation with MuPHI shows that vision‑language models can ground individual modalities but struggle to bind evidence into a coherent harm mechanism, and that existing harm detection methods transfer poorly to new harm distributions. To address this, the authors propose Compositional Harm Interpretation Learning (CHIL), which rewards evidence identification, cross‑modal binding, and verdict consistency, leading to significant improvements in out‑of‑distribution transfer and counterfactual robustness.
Hugging Face Trending Papers
Jun 25
HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models
Large vision-language models (LVLMs) have recently shown immense potential in automated content moderation, sparking growing interest in developing harmful-video benchmarks. However, we identify two primary limitations in existing works: 1) The multi-layered characteristics of harmful videos are overlooked.
arXiv AI
Sep 7
When Seeing Overrides Knowing: Visual Dominance and Deferral-Based Method for Personalized Safety in VLMs
The paper introduces MPS-Bench, a benchmark of 5,181 scenarios from 584 real-world images across 12 high-risk domains, each paired with a hidden user profile, to evaluate personalized safety in vision‑language models (VLMs). Eight leading VLMs were tested and found to almost always respond directly (86‑99%) without seeking missing context, scoring no higher than 2.6/5 on personalized safety. The authors identify a phenomenon called visual dominance, where visual information enters text representations early and suppresses textual risk signals, and propose PRISM, a lightweight input monitor that predicts when a query should be deferred, achieving 0.978 AUC and outperforming all tested models on the safety‑utility Pareto frontier.
arXiv AI
Sep 10
AV-SafetyBench: A Safety Benchmark for Text-to-Audio-Video Generation
arXiv:2609.06991v1 Announce Type: cross Abstract: Recent text-to-audio-video (T2AV) models jointly generate video, speech, sound effects, and ambience from a single text prompt. This capability poses...
arXiv AI
Aug 28
Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation
Multi2AV‑Safety is a new benchmark for evaluating safety in multimodal-to-audio‑video generation. It covers all 11 non‑singleton conditioning configurations (text, image, audio, video) and contains 11,024 attack instances. The benchmark reveals that safety guards often fail when harmful semantics arise from combinations of benign inputs or when explicit harmful cues are masked by benign multimodal context, highlighting a gap in compositional risk perception.
arXiv AI
Oct 19
Harmful YouTube Video Detection: A Taxonomy of Online Harm and MLLMs as Alternative Annotators
The paper presents a new taxonomy for online harm on video platforms, expanding existing frameworks to multimodal contexts and defining six content-based categories: Information, Hate and harassment, Addictive, Clickbait, Sexual, and Physical harms. It also evaluates multimodal large language models (MLLMs) as alternative annotators, comparing GPT‑4‑Turbo, non‑trained crowdworkers, and trained human annotators on a dataset of 19,422 potentially harmful YouTube videos. The study finds that MLLMs can perform comparably to human annotators in both binary and multi‑label classification tasks, suggesting potential applications in content moderation and social science research.