The paper presents a new taxonomy for online harm on video platforms, expanding existing frameworks to multimodal contexts and defining six content-based categories: Information, Hate and harassment, Addictive, Clickbait, Sexual, and Physical harms. It also evaluates multimodal large language models (MLLMs) as alternative annotators, comparing GPT‑4‑Turbo, non‑trained crowdworkers, and trained human annotators on a dataset of 19,422 potentially harmful YouTube videos. The study finds that MLLMs can perform comparably to human annotators in both binary and multi‑label classification tasks, suggesting potential applications in content moderation and social science research.
By Claire Wonjeong Jo, Miki Weso{\l}owska, Magdalena Wojcieszak
arXiv:2606. 09700v1 Announce Type: cross Abstract: Large language model (LLM)-powered content moderation systems have become a critical defense against harmful online content.
By Qin Yang, Lu Malloy, Joshua Lee, Xiaohan Chang, Meisam Mohammady, Doowon Kim, Yuan Hong
The paper introduces Bot‑Mod, a moderation framework that detects malicious intent in multi‑agent systems by interrogating agents with diagnostic probes and updating a Bayesian posterior over candidate intents. Bot‑Mod starts with a content‑only prior and refines its inference using moderator‑assigned likelihoods until a stopping criterion is reached. Evaluations on the Molt‑Intent dataset, an external social engineering dataset, and three agent LLMs show that Bot‑Mod outperforms content‑only moderators, a fine‑tuned classifier, and judge‑based interrogators, achieving higher F1 scores both in‑distribution and out‑of‑distribution.
By Ali Al-Lawati, Nafis Tripto, Abolfazl Ansari, Fali Wang, Jason Lucas, Suhang Wang, Dongwon Lee
EvoHarmBench is a dynamic adversarial evaluation framework that simulates how users iteratively modify harmful content to evade moderation. It uses an optimization loop that evolves evasion strategies at the semantic-cluster level while maintaining human readability, and tests 229 semantic sub-clusters across five violation categories derived from 5,002 real-world adversarial samples. The study shows that even state‑of‑the‑art LLM‑based moderators can be bypassed with an 80.3% success rate after twelve iterations, highlighting significant vulnerabilities in current systems.
By Ruijie Jian, Benlei Cui, Ting Ma, Haidong Ding, Kangwei Liu, Ziwen Xu, Longtao Huang, Hui Xue, Ziqiang Zhu, Junjie Li, Haiwen Hong
arXiv:2609.22696v1 Announce Type: new
Abstract: Decentralized social media platforms create new opportunities and challenges for computational mental health research because data access, moderation,...
By Gaurab Chhetri, Anandi Dutta, Subasish Das
The study audits hate‑speech moderation on Twitter (now X) using 540,000 annotated tweets from a full day. Eighty percent of hateful tweets, including violent content, remained online after five months, and removal was only slightly more likely than for non‑hateful tweets, far below the rates for scams or adult content. Automated detection could not reliably classify hate but ranked it highly, allowing human triage; however, current staffing curbed little exposure, while substantial reductions were financially feasible and far below applicable regulatory fines.
By Manuel Tonneau, Dylan Thurgood, Diyi Liu, Niyati Malhotra, Victor Orozco-Olvera, Ralph Schroeder, Scott A. Hale, Manoel Horta Ribeiro, Paul R\"ottger, Samuel P. Fraiberger
arXiv:2602. 02838v2 Announce Type: replace-cross Abstract: The detection of online influence operations -- coordinated campaigns by malicious actors to spread narratives -- has traditionally depended on content analysis or network features.
By Philipp J. Schneider, Lanqin Yuan, Marian-Andrei Rizoiu
arXiv:2606. 30801v1 Announce Type: cross Abstract: Personalization algorithms determine what content users encounter on online platforms.
By Alessandro Morosini, Sarah H. Cen, Andrew Ilyas, Hedi Driss, Aleksander M\k{a}dry, Chara Podimata
arXiv:2606. 04867v1 Announce Type: new Abstract: As AI companion platforms such as Replika and Character.
By Yanjing Ren, Reza Ebrahimi, TengTeng Ma
The study investigates how the demographic makeup of crowdsourced moderators influences who is protected from perceived toxic content online. Using data from 16,221 U.S. respondents who evaluated over 100,000 comments from Twitter, Reddit, and 4chan, the authors find that moderators tend to protect users who share their own demographic identities, a pattern that is amplified when moderator pools mirror the demographics of Prolific participants. Even fully representative moderator groups fail to provide equal protection, leaving Black and LGB users underprotected unless they are overrepresented.
By Zhaodi Chen, Byungkyu Lee
The paper "Queering the Audits: Community-Based Auditing of AI Harms to Queer Communities" reports that AI systems often embed majority-group defaults, leading to harms such as erasure, misclassification, and discrimination against queer communities. Traditional technical audits, which rely on aggregate metrics, fail to detect these harms that become apparent only through lived experience. The authors conducted a participatory auditing workshop at EurIPS 2025, where 16 queer community members used the 4Cs harm taxonomy to audit four AI case studies, tracing harms to problem definition, data collection, and contractual structures that exploit vulnerable populations.
By Organizers of QueerInAI, :, A Pranav, Alissa A. Valentine, Alex Markham, Beckett LeClair, Tereza Blazkova, Ekaterina Kornilitsina, Sofie H. Bruun, Gerasimos Spanakis, Anne Lauscher
arXiv:2608. 14692v1 Announce Type: cross Abstract: Personalized, generative AI systems increasingly adapt their behavior to individual users over time, fundamentally changing model behavior.
By Hannah Cha