arXiv Computation and Language

The Enforcement and Feasibility of Hate Speech Moderation

The study audits hate‑speech moderation on Twitter (now X) using 540,000 annotated tweets from a full day. Eighty percent of hateful tweets, including violent content, remained online after five months, and removal was only slightly more likely than for non‑hateful tweets, far below the rates for scams or adult content. Automated detection could not reliably classify hate but ranked it highly, allowing human triage; however, current staffing curbed little exposure, while substantial reductions were financially feasible and far below applicable regulatory fines.

arXiv Computation and Language
Sep 1

When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech

The paper introduces WSF-ARG+, a new dataset that pairs hate speech with check‑worthiness annotations, and presents an LLM‑in‑the‑loop framework to streamline the annotation process. Experiments with 12 open‑weight large language models demonstrate that the framework cuts human effort while maintaining annotation quality. The study also shows that incorporating check‑worthiness labels improves hate‑speech detection performance, boosting macro‑F1 scores for large models by up to 0.213 and averaging 0.154 across models.

By Nicol\'as Benjam\'in Ocampo, Tommaso Caselli, Davide Ceolin
Hugging Face Trending Papers
Aug 19

Introducing the Privacy-HSD Trade-off: Hate Speech Detection, but not at the Cost of Privacy

The paper introduces the privacy‑HSD trade‑off, highlighting that automatic hate speech detection systems can inadvertently compromise user privacy by encoding authorship. It demonstrates that such systems may achieve high performance at the expense of privacy, and proposes a new domain‑specific technique, AgnoSpeech, alongside other text privatization methods to balance these competing goals. The authors benchmark these methods, showing that while challenging, it is feasible to protect privacy without sacrificing hate‑speech detection effectiveness.

arXiv AI
Sep 12

Characterizing Bluesky Content Moderation Service: From Automation of Service to Landscape of Harms

The study audits Bluesky’s Moderation Service (BMS) using its 10.6 million public moderation labels from 2025. It finds that BMS operates as a human‑AI collaboration: sexual and graphic content is flagged automatically in seconds, while more nuanced or high‑stakes content requires human review that can take hours or days. The system shows high precision (0.837) but low recall (0.222), with annotators detecting 4.5 times more harmful content than the system, and clustering reveals harms ranging from hostility toward protected groups to the spread of explicit material.

By Pushpdeep Singh, Sayeh Jarollahi, Ayan Majumdar, Vabuk Pahari, Abhijnan Chakraborty, Krishna P. Gummadi, Ingmar Weber, Abhisek Dash
arXiv Computation and Language
Aug 25

Whitewashing Hate, Smearing Harmless Content: Annotator-Style Rebuttal Attacks on LLM-Based Moderation

The paper investigates how annotator-style rebuttals can manipulate large language model (LLM) moderation systems, either by whitewashing hateful content as normal or smearing normal content as hateful. Using a rejudge protocol that adds decision‑boundary perturbations and adversarial rationales, the authors show that such rebuttals significantly degrade moderation performance, especially in multi‑turn settings. The study finds consistent, model‑specific asymmetries between the two manipulation directions and demonstrates that explicit reasoning prompts and defensive instructions mitigate but do not eliminate the vulnerability.

By Junyu Lu, Kaiyuan Liu, Jingyi Kang, Deyi Ji, Hailong Zhang, Lanyun Zhu, Qi Zhu, Bo Xu, Liang Yang, Hongfei Lin
arXiv Computation and Language
Aug 27

From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation

The paper presents an instruction‑tuned large language model (LLM) based on Qwen3 that is fine‑tuned for hate speech mitigation by unifying 36 English hate speech datasets. The authors show that this generalist LLM achieves state‑of‑the‑art performance on in‑domain benchmarks and delivers significant gains in cross‑domain and cross‑lingual generalization, outperforming specialist encoder‑based classifiers.

By Lukas Edman, Daryna Dementieva, Alexander Fraser
arXiv Computation and Language
Sep 11

Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection

The paper explores using a large language model (LLM) to generate background context for social media posts and tests four ways to integrate this context into a Sentence-BERT (SBERT) hate‑speech detection classifier. The methods include text concatenation, embedding concatenation, hierarchical transformer fusion, and LLM‑driven text enhancement. Experiments on the Latent Hatred dataset of implicitly hateful tweets and the MAMI dataset of misogynous memes show that adding generated context can raise F1 scores by up to 3 points in textual and 6 points in multimodal settings compared to a zero‑context baseline, with embedding concatenation yielding the best results.

By Joshua Wolfe Brook, Ilia Markov