arXiv AI By Somaiyeh Dehghan, Mehmet Umut Sen, Berrin Yanikoglu

Dealing with Annotator Disagreement in Hate Speech Classification

Read the original on arXiv AI →

arXiv:2502. 08266v3 Announce Type: replace-cross Abstract: Hate speech detection is a crucial task, especially on social media where harmful content can spread quickly.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Sep 1

When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech

The paper introduces WSF-ARG+, a new dataset that pairs hate speech with check‑worthiness annotations, and presents an LLM‑in‑the‑loop framework to streamline the annotation process. Experiments with 12 open‑weight large language models demonstrate that the framework cuts human effort while maintaining annotation quality. The study also shows that incorporating check‑worthiness labels improves hate‑speech detection performance, boosting macro‑F1 scores for large models by up to 0.213 and averaging 0.154 across models.

By Nicol\'as Benjam\'in Ocampo, Tommaso Caselli, Davide Ceolin
arXiv Machine Learning
Jun 9

Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learning

arXiv:2009. 10277v2 Announce Type: replace-cross Abstract: We propose a system for measuring hate speech on a continuous, interval-valued spectrum ranging from genocidal to supportive speech by combining supervised deep learning with faceted Rasch item response theory (IRT).

By Chris J. Kennedy, Geoff Bacon, Alexander Sahn, Claudia von Vacano
arXiv AI
Sep 25

An Explainable DistilBERT-BiLSTM-Attention Framework for Binary and Multi-Class Hate Speech Detection

The paper presents an explainable hate‑speech detection framework that combines DistilBERT embeddings, a Bi‑LSTM network, and an attention mechanism to capture contextual and sequential information. It uses LIME to highlight influential text features, providing transparency in predictions. Evaluated on two benchmark datasets for both binary and multi‑class tasks, the model achieves F1‑scores of 96.78%–99.53% for binary classification and 94.99%–97.00% for multi‑class classification, outperforming existing baselines.

By Rameesha Zia, Muhammad Shahid Iqbal Malik
arXiv Computation and Language
Aug 25

Whitewashing Hate, Smearing Harmless Content: Annotator-Style Rebuttal Attacks on LLM-Based Moderation

The paper investigates how annotator-style rebuttals can manipulate large language model (LLM) moderation systems, either by whitewashing hateful content as normal or smearing normal content as hateful. Using a rejudge protocol that adds decision‑boundary perturbations and adversarial rationales, the authors show that such rebuttals significantly degrade moderation performance, especially in multi‑turn settings. The study finds consistent, model‑specific asymmetries between the two manipulation directions and demonstrates that explicit reasoning prompts and defensive instructions mitigate but do not eliminate the vulnerability.

By Junyu Lu, Kaiyuan Liu, Jingyi Kang, Deyi Ji, Hailong Zhang, Lanyun Zhu, Qi Zhu, Bo Xu, Liang Yang, Hongfei Lin
arXiv AI
Aug 28

Training-Time Explainability for Multilingual Hate Speech Detection: Aligning Model Reasoning with Human Rationales

The paper proposes a training‑time explainability framework that aligns model reasoning with human‑annotated rationales to improve both classification performance and interpretability for multilingual hate speech detection. It is evaluated on HateXplain (English) and BullySent (Hinglish), datasets that capture anti‑Muslim hate in culturally coded, multilingual forms. Using methods such as LIME, Integrated Gradients, Grad‑X‑Input, and attention, the study shows that gradient‑ and attention‑based regularization boosts F‑scores, enhances plausibility and faithfulness, and captures culturally specific cues for detecting implicit anti‑Muslim hate.

By Muhammad Deedahwar Mazhar Qureshi, Sannaan Khan, Muhammad Atif Qureshi, Wael Rashwan