Dealing with Annotator Disagreement in Hate Speech Classification
arXiv:2502. 08266v3 Announce Type: replace-cross Abstract: Hate speech detection is a crucial task, especially on social media where harmful content can spread quickly.
The paper "HateSense: Hierarchical Sentiment Analysis Framework for Hate Speech Detection: Implementing Binary and Multiclass Classification Strategy" revisits a transformer-based framework that combines hierarchical classification and sentiment features for hate speech detection. Experiments on the Davidson and HateXplain datasets show that a dominant-class-first hierarchy actually lowers macro‑F1, sentiment features have minimal impact, and class‑weighted training yields the largest performance gains. Cross‑dataset transfer and slur reuse reveal that in‑domain metrics overstate real‑world reliability.
arXiv:2502. 08266v3 Announce Type: replace-cross Abstract: Hate speech detection is a crucial task, especially on social media where harmful content can spread quickly.
The paper presents an explainable hate‑speech detection framework that combines DistilBERT embeddings, a Bi‑LSTM network, and an attention mechanism to capture contextual and sequential information. It uses LIME to highlight influential text features, providing transparency in predictions. Evaluated on two benchmark datasets for both binary and multi‑class tasks, the model achieves F1‑scores of 96.78%–99.53% for binary classification and 94.99%–97.00% for multi‑class classification, outperforming existing baselines.
arXiv:2610.16957v1 Announce Type: cross Abstract: Standard hate speech detection datasets often emphasize explicit harmful language, but directed harm can also be expressed indirectly, without clearl...
The paper evaluates six structured decision models for hate‑speech moderation across four datasets, comparing them to specialized, zero‑shot, commercial, and supervised baselines. It investigates whether providing dataset definitions or decomposing them into multiple questions improves classification, and measures latency and cost. Results show that commercial LLMs outperform decision models on only one dataset, definitions alter up to 28% of predictions without consistent gains, and decomposition helps in only 20% of cases, yet the best hosted decision model approaches the best commercial LLM’s performance with roughly 97% lower inference cost.
The paper introduces WSF-ARG+, a new dataset that pairs hate speech with check‑worthiness annotations, and presents an LLM‑in‑the‑loop framework to streamline the annotation process. Experiments with 12 open‑weight large language models demonstrate that the framework cuts human effort while maintaining annotation quality. The study also shows that incorporating check‑worthiness labels improves hate‑speech detection performance, boosting macro‑F1 scores for large models by up to 0.213 and averaging 0.154 across models.
arXiv:2507. 10177v2 Announce Type: replace-cross Abstract: Although Large Language Models (LLMs) have demonstrated significant advancements in natural language processing tasks, their effectiveness in the classification and transformation of abusive text into non-abusive versions remains an area for exploration.
The study evaluates how well three sentiment‑analysis tools (TextBlob, VADER, Twitter‑roBERTa‑base) and three large language models (Qwen3‑32B, GPT‑OSS‑120B, Llama‑4‑Maverick‑17B) agree with six human raters on 100 tweets. Agreement was measured with Cohen’s and Fleiss’ kappa, revealing only fair inter‑human agreement and higher concordance for binary sentiment labels than for three‑class labels. Twitter‑roBERTa‑base achieved the strongest alignment with humans, especially for negative versus non‑negative sentiment, while the LLMs showed substantial agreement among themselves and moderate to substantial alignment with humans, particularly for positive versus non‑positive classifications. The findings emphasize that domain‑specific fine‑tuning and human‑centered evaluation are essential for reliable social media sentiment analysis.
The paper presents an instruction‑tuned large language model (LLM) based on Qwen3 that is fine‑tuned for hate speech mitigation by unifying 36 English hate speech datasets. The authors show that this generalist LLM achieves state‑of‑the‑art performance on in‑domain benchmarks and delivers significant gains in cross‑domain and cross‑lingual generalization, outperforming specialist encoder‑based classifiers.
arXiv:2606. 08770v1 Announce Type: cross Abstract: The analysis of internet memes in the Nepali language is complicated by frequent code-mixing and a lack of established baseline resources.
arXiv:2607. 20447v1 Announce Type: cross Abstract: This paper describes our system for the EEUCA 2026 Shared Task on toxicity classification in gaming chat.
Large language models (LLMs) demonstrate impressive performance across a wide range of general NLP tasks; however, their effectiveness in sensitive domains, such as hate speech detection, remains less...
arXiv:2610.28337v1 Announce Type: cross Abstract: Emoji are widely treated as compact affective tokens whose meaning is summarised by a single sentiment score. We argue that this picture is incomplet...