arXiv AI

Towards Safer Social Media Platforms: Scalable and Performant Few-Shot Harmful Content Moderation Using Large Language Models

The paper presents a scalable approach to harmful content moderation on social media by leveraging large language models (LLMs) for few-shot, in-context learning. Experiments across multiple LLMs show that this method outperforms proprietary baselines such as Perspective and OpenAI Moderation, as well as prior few-shot learning techniques, in detecting harmful content. The study also explores the addition of visual cues like video thumbnails to assess multimodal improvements, highlighting the advantages of LLM-based moderation for dynamic and large-scale content filtering.

arXiv AI

Oct 19

Harmful YouTube Video Detection: A Taxonomy of Online Harm and MLLMs as Alternative Annotators

The paper presents a new taxonomy for online harm on video platforms, expanding existing frameworks to multimodal contexts and defining six content-based categories: Information, Hate and harassment, Addictive, Clickbait, Sexual, and Physical harms. It also evaluates multimodal large language models (MLLMs) as alternative annotators, comparing GPT‑4‑Turbo, non‑trained crowdworkers, and trained human annotators on a dataset of 19,422 potentially harmful YouTube videos. The study finds that MLLMs can perform comparably to human annotators in both binary and multi‑label classification tasks, suggesting potential applications in content moderation and social science research.

By Claire Wonjeong Jo, Miki Weso{\l}owska, Magdalena Wojcieszak

llms multimodal

arXiv AI

Jul 14

Tool-MCoT: Tool Augmented Multimodal Chain-of-Thought for Content Safety Moderation

arXiv:2604. 06205v2 Announce Type: replace-cross Abstract: The growth of online platforms and user content requires strong content moderation systems that can handle complex inputs from various media types.

By Shutong Zhang, Dylan Zhou, Yinxiao Liu, Yang Yang, Huiwen Luo, Wenfei Zou

llms fine-tuning multimodal safety

Hugging Face Trending Papers

Jun 25

HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models

Large vision-language models (LVLMs) have recently shown immense potential in automated content moderation, sparking growing interest in developing harmful-video benchmarks. However, we identify two primary limitations in existing works: 1) The multi-layered characteristics of harmful videos are overlooked.

llms multimodal benchmarks

arXiv AI

Jul 31

TANDEM: Temporal-Aware Neural Detection for Multimodal Hate Speech

arXiv:2601. 11178v3 Announce Type: replace Abstract: Social media platforms are increasingly dominated by long-form multimodal content, where harmful narratives are constructed through a complex interplay of audio, visual, and textual cues.

By Girish A. Koushik, Helen Treharne, Diptesh Kanojia

llms reinforcement-learning multimodal benchmarks safety

Hugging Face Trending Papers

Sep 9

MUCnoHARM@GermEval Shared Task 2026: Retrieval-based In-Context Learning for Defamatory Offences, and Where It Falls Short

With hate speech being ubiquitous online, automatic detection is crucial, in particular when it comes to criminally relevant social media posts. We study a variety of retrieval-based in-context learni...

llms agents

Hugging Face Trending Papers

Jul 29

AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes

Hateful memes are a growing form of multimodal online harm, where hostile intent is often conveyed through the joint interpretation of images, text, cultural references, and implicit targets. While hateful meme detection has advanced in high-resource languages, Arabic remains underexplored, with existing meme resources focusing mainly on propaganda or coarse harmful-content labels.

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 16

YTClickbait21K: Human-Annotated Multimodal Dataset for YouTube Clickbait Detection Across Diverse Channels and Content Categories

arXiv:2606. 14780v1 Announce Type: cross Abstract: Clickbait content on video-sharing platforms poses a significant challenge to information reliability, yet progress in automated detection has been constrained by the lack of large-scale, high-quality multimodal datasets.

By Md. Minhazul Islam, Md. Tanbeer Jubaer, Amith Khandakar, Shovon Sarker, Sumaiya Rahman, Md. Masum Mia, Mohamed Arselene Ayari, Hamed Noori

multimodal benchmarks

arXiv Computation and Language

Sep 11

Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection

The paper explores using a large language model (LLM) to generate background context for social media posts and tests four ways to integrate this context into a Sentence-BERT (SBERT) hate‑speech detection classifier. The methods include text concatenation, embedding concatenation, hierarchical transformer fusion, and LLM‑driven text enhancement. Experiments on the Latent Hatred dataset of implicitly hateful tweets and the MAMI dataset of misogynous memes show that adding generated context can raise F1 scores by up to 3 points in textual and 6 points in multimodal settings compared to a zero‑context baseline, with embedding concatenation yielding the best results.

By Joshua Wolfe Brook, Ilia Markov

llms rag multimodal

arXiv AI

Sep 30

Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?

The paper investigates whether multimodal large language models (MLLMs) can generate and detect realistic multimodal fake news on social media. Using a multi‑agent framework—comprising a story agent, an image agent, and a critic agent—the authors produced over 9,000 paired multimodal news posts across science, health, and entertainment domains. They benchmarked 16 open‑ and closed‑source MLLMs for automated detection and found that most models fall far short of human accuracy, especially in identifying image authenticity, highlighting the need for stronger defenses against social media fake news.

By Jiyao Yang, Yang Liu, Zhenyue Qin, Qingyu Chen, Xiuzhen Zhang

llms agents multimodal benchmarks

arXiv Computation and Language

Sep 10

MUCnoHARM@GermEval Shared Task 2026: Retrieval-based In-Context Learning for Defamatory Offences, and Where It Falls Short

arXiv:2609.09791v1 Announce Type: new Abstract: With hate speech being ubiquitous online, automatic detection is crucial, in particular when it comes to criminally relevant social media posts. We stu...

By Kristin Gnadt, Maximilian Meidinger, Matthias A{\ss}enmacher

llms agents

arXiv AI

Jun 4

Dynamic Content Moderation in Livestreams: Combining Supervised Classification with MLLM-Boosted Similarity Matching

arXiv:2512. 03553v3 Announce Type: replace-cross Abstract: Content moderation remains a critical yet challenging task for large-scale user-generated video platforms, especially in livestreaming environments where moderation must be timely, multimodal, and robust to evolving forms of unwanted content.

By Wei Chee Yew, Hailun Xu, Sanjay Saha, Xiaotian Fan, Hiok Hian Ong, David Yuchen Wang, Kanchan Sarkar, Zhenheng Yang, Danhui Guan

llms multimodal safety

arXiv Machine Learning

Jun 17

Moderating Illicit Online Image Promotion for Unsafe User-Generated Content Games Using Large Vision-Language Models

arXiv:2403. 18957v3 Announce Type: replace-cross Abstract: Online user generated content games (UGCGs) are increasingly popular among children and adolescents for social interaction and more creative online entertainment.

By Keyan Guo, Ayush Utkarsh, Wenbo Ding, Isabelle Ondracek, Ziming Zhao, Guo Freeman, Nishant Vishwamitra, Hongxin Hu

llms fine-tuning multimodal safety

Related stories