The paper analyzes how malicious actors create and share synthetic non-consensual intimate images (SNCII) across online forums, focusing on 4chan and Reddit. By examining over 280,000 4chan comments and 78,000 Reddit posts, the study identifies a wide array of resources—including 156 LoRA files targeting 141 female celebrities and 54 commercial nudification apps—used by both highly technical and less sophisticated users. The findings show that knowledge transfer between experts and newcomers fuels the spread of these illicit resources, revealing gaps in current regulatory frameworks and suggesting intervention points to deter such activity.
By Bernardo B. P. Medeiros (University of Florida), Malvika Jadhav (University of Florida), Allison Lu (University of Florida), Tadayoshi Kohno (Georgetown University), Vincent Bindschaedler (University of Florida), Kevin R. B. Butler (University of Florida)
The study audits Bluesky’s Moderation Service (BMS) using its 10.6 million public moderation labels from 2025. It finds that BMS operates as a human‑AI collaboration: sexual and graphic content is flagged automatically in seconds, while more nuanced or high‑stakes content requires human review that can take hours or days. The system shows high precision (0.837) but low recall (0.222), with annotators detecting 4.5 times more harmful content than the system, and clustering reveals harms ranging from hostility toward protected groups to the spread of explicit material.
By Pushpdeep Singh, Sayeh Jarollahi, Ayan Majumdar, Vabuk Pahari, Abhijnan Chakraborty, Krishna P. Gummadi, Ingmar Weber, Abhisek Dash
The study audits hate‑speech moderation on Twitter (now X) using 540,000 annotated tweets from a full day. Eighty percent of hateful tweets, including violent content, remained online after five months, and removal was only slightly more likely than for non‑hateful tweets, far below the rates for scams or adult content. Automated detection could not reliably classify hate but ranked it highly, allowing human triage; however, current staffing curbed little exposure, while substantial reductions were financially feasible and far below applicable regulatory fines.
By Manuel Tonneau, Dylan Thurgood, Diyi Liu, Niyati Malhotra, Victor Orozco-Olvera, Ralph Schroeder, Scott A. Hale, Manoel Horta Ribeiro, Paul R\"ottger, Samuel P. Fraiberger
The paper investigates how large language models (LLMs) used as labelers for toxic content differ from human crowd labels, focusing on the bias that LLMs, particularly Claude Haiku, over‑flag second‑person address (“you”) as toxic. On a dataset of 1.25 million news comments, Haiku incorrectly flags a much higher proportion of non‑toxic comments containing “you” compared to human raters, leading to a false increase in measured toxicity with reply depth. The authors show that rewriting “you” in the third person removes most of this bias, and that a certification approach—estimating laws rather than trusting individual labelers—can produce unbiased toxicity metrics even when using LLM scores.
"Why it matters": The study reveals that relying on LLM labelers without correcting for their bias can distort our understanding of toxic discourse online, potentially leading to misguided moderation policies.
By Liang Yue, Eric A. Vance
arXiv:2602. 02838v2 Announce Type: replace-cross Abstract: The detection of online influence operations -- coordinated campaigns by malicious actors to spread narratives -- has traditionally depended on content analysis or network features.
By Philipp J. Schneider, Lanqin Yuan, Marian-Andrei Rizoiu
arXiv:2607. 26236v1 Announce Type: cross Abstract: AI-generated counterspeech offers a scalable and effective strategy to mitigate online toxicity by promoting more constructive dialogue.
By Lorenzo Cima, Alessio Miaschi, Amaury Trujillo, Marco Avenuti, Felice Dell'Orletta, Stefano Cresci