The paper analyzes how malicious actors create and share synthetic non-consensual intimate images (SNCII) across online forums, focusing on 4chan and Reddit. By examining over 280,000 4chan comments and 78,000 Reddit posts, the study identifies a wide array of resources—including 156 LoRA files targeting 141 female celebrities and 54 commercial nudification apps—used by both highly technical and less sophisticated users. The findings show that knowledge transfer between experts and newcomers fuels the spread of these illicit resources, revealing gaps in current regulatory frameworks and suggesting intervention points to deter such activity.
arXiv:2606. 27234v1 Announce Type: cross Abstract: AI nudification uses generative models to create synthetic non-consensual sexually explicit imagery (SNEACI) of real individuals.
arXiv:2602. 02838v2 Announce Type: replace-cross Abstract: The detection of online influence operations -- coordinated campaigns by malicious actors to spread narratives -- has traditionally depended on content analysis or network features.
The study investigates how the demographic makeup of crowdsourced moderators influences who is protected from perceived toxic content online. Using data from 16,221 U.S. respondents who evaluated over 100,000 comments from Twitter, Reddit, and 4chan, the authors find that moderators tend to protect users who share their own demographic identities, a pattern that is amplified when moderator pools mirror the demographics of Prolific participants. Even fully representative moderator groups fail to provide equal protection, leaving Black and LGB users underprotected unless they are overrepresented.
arXiv:2606. 05256v1 Announce Type: new Abstract: This study analyzes a publicly released dataset from a discontinued field experiment on Reddit's r/ChangeMyView.
The study audits Bluesky’s Moderation Service (BMS) using its 10.6 million public moderation labels from 2025. It finds that BMS operates as a human‑AI collaboration: sexual and graphic content is flagged automatically in seconds, while more nuanced or high‑stakes content requires human review that can take hours or days. The system shows high precision (0.837) but low recall (0.222), with annotators detecting 4.5 times more harmful content than the system, and clustering reveals harms ranging from hostility toward protected groups to the spread of explicit material.
arXiv:2610.19267v1 Announce Type: cross Abstract: Online memes are often carriers of unsafe content such as disinformation and propaganda. Their rapid evolution in response to news events and strong...
arXiv:2607. 15267v1 Announce Type: new Abstract: Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate.
The paper reports the first large‑scale empirical comparison of AI‑agent and human online communities, analyzing 73,899 Moltbook and 189,838 Reddit posts across five matched communities. It finds that Moltbook shows extreme participation inequality (Gini = 0.84 vs. 0.47) and high cross‑community author overlap (33.8% vs. 0.5%). Linguistically, AI‑generated content is emotionally flattened, more assertive than exploratory, and socially detached, leading to community‑level homogenization that is largely a structural artifact of shared authorship. At the individual level, AI agents are more identifiable than human users due to outlier stylistic profiles amplified by their extreme posting volume.
The paper presents an agentic framework for detecting conspiratorial content in social media by inferring the speaker’s intent rather than merely identifying explicit claims. It leverages social context and adaptive tool use, demonstrating superior performance over text-only and non-agentic models on a large Hebrew tweet dataset spanning election cycles and the COVID pandemic. The study highlights the importance of context-aware, reasoning-driven approaches for accurate conspiracy detection.
The paper presents a new taxonomy for online harm on video platforms, expanding existing frameworks to multimodal contexts and defining six content-based categories: Information, Hate and harassment, Addictive, Clickbait, Sexual, and Physical harms. It also evaluates multimodal large language models (MLLMs) as alternative annotators, comparing GPT‑4‑Turbo, non‑trained crowdworkers, and trained human annotators on a dataset of 19,422 potentially harmful YouTube videos. The study finds that MLLMs can perform comparably to human annotators in both binary and multi‑label classification tasks, suggesting potential applications in content moderation and social science research.
arXiv:2607. 18263v1 Announce Type: new Abstract: AI-generated non-consensual intimate imagery (AIG-NCII) is not adequately addressed in AI/ML literature regarding AI-generated media, commonly referred to as "deepfakes".
arXiv:2403. 18957v3 Announce Type: replace-cross Abstract: Online user generated content games (UGCGs) are increasingly popular among children and adolescents for social interaction and more creative online entertainment.