Dealing with Annotator Disagreement in Hate Speech Classification
arXiv:2502. 08266v3 Announce Type: replace-cross Abstract: Hate speech detection is a crucial task, especially on social media where harmful content can spread quickly.
arXiv:2009. 10277v2 Announce Type: replace-cross Abstract: We propose a system for measuring hate speech on a continuous, interval-valued spectrum ranging from genocidal to supportive speech by combining supervised deep learning with faceted Rasch item response theory (IRT).
arXiv:2502. 08266v3 Announce Type: replace-cross Abstract: Hate speech detection is a crucial task, especially on social media where harmful content can spread quickly.
arXiv:2601. 11178v3 Announce Type: replace Abstract: Social media platforms are increasingly dominated by long-form multimodal content, where harmful narratives are constructed through a complex interplay of audio, visual, and textual cues.
The paper presents an explainable hate‑speech detection framework that combines DistilBERT embeddings, a Bi‑LSTM network, and an attention mechanism to capture contextual and sequential information. It uses LIME to highlight influential text features, providing transparency in predictions. Evaluated on two benchmark datasets for both binary and multi‑class tasks, the model achieves F1‑scores of 96.78%–99.53% for binary classification and 94.99%–97.00% for multi‑class classification, outperforming existing baselines.
The paper introduces FAID, a fine‑grained adaptive framework for detecting implicit hate speech. It first classifies samples into Shallow, Targeted, or Context‑Dependent categories and then applies tailored strategies—prompt‑tuning for shallow cases, knowledge augmentation for targeted ones, and an agentic prompt‑generation system for context‑dependent posts. Experiments on four benchmark datasets show that FAID outperforms state‑of‑the‑art baselines by allocating computational effort only where needed.
arXiv:2606. 18852v1 Announce Type: cross Abstract: Classifying implicit hate speech remains a challenge, as intent is often masked through insinuation and context rather than explicit slurs.
The paper presents an instruction‑tuned large language model (LLM) based on Qwen3 that is fine‑tuned for hate speech mitigation by unifying 36 English hate speech datasets. The authors show that this generalist LLM achieves state‑of‑the‑art performance on in‑domain benchmarks and delivers significant gains in cross‑domain and cross‑lingual generalization, outperforming specialist encoder‑based classifiers.
The paper proposes a training‑time explainability framework that aligns model reasoning with human‑annotated rationales to improve both classification performance and interpretability for multilingual hate speech detection. It is evaluated on HateXplain (English) and BullySent (Hinglish), datasets that capture anti‑Muslim hate in culturally coded, multilingual forms. Using methods such as LIME, Integrated Gradients, Grad‑X‑Input, and attention, the study shows that gradient‑ and attention‑based regularization boosts F‑scores, enhances plausibility and faithfulness, and captures culturally specific cues for detecting implicit anti‑Muslim hate.
The paper introduces WSF-ARG+, a new dataset that pairs hate speech with check‑worthiness annotations, and presents an LLM‑in‑the‑loop framework to streamline the annotation process. Experiments with 12 open‑weight large language models demonstrate that the framework cuts human effort while maintaining annotation quality. The study also shows that incorporating check‑worthiness labels improves hate‑speech detection performance, boosting macro‑F1 scores for large models by up to 0.213 and averaging 0.154 across models.
Large language models (LLMs) demonstrate impressive performance across a wide range of general NLP tasks; however, their effectiveness in sensitive domains, such as hate speech detection, remains less...
The paper examines how four leading vision‑language models—LLaVA‑7B, Qwen‑VL, GPT‑4o mini, and Claude 3 Haiku—perform in detecting hateful content within memes. It evaluates the models under zero‑shot and few‑shot prompting, focusing not only on classification accuracy but also on the qualitative justifications they generate. The study highlights that these models often overlook contextual nuances, irony, and subtle cues essential for accurately identifying hate speech in memes.
The paper discusses how large language models (LLMs) are used in various evaluation roles—examining benchmarks, judging other models, and rating human content—and frames each as a measurement problem. It proposes using Rasch measurement theory (RMT) to decompose ordinal ratings into distinct facets on a common scale, offering diagnostics for miscalibration and rater bias. A case study applying RMT to the Measuring Hate Speech corpus reveals systematic differences between LLMs and human raters in severity, calibration, robustness, sensitivity, and scale use, suggesting RMT should be part of the evaluation toolkit for LLMs in all roles.
arXiv:2601.06631v2 Announce Type: replace Abstract: Building NLP systems for subjective tasks requires one to ensure their alignment to contrasting human values. We propose the MultiCalibrated Subjec...