The paper "HateSense: Hierarchical Sentiment Analysis Framework for Hate Speech Detection: Implementing Binary and Multiclass Classification Strategy" revisits a transformer-based framework that combines hierarchical classification and sentiment features for hate speech detection. Experiments on the Davidson and HateXplain datasets show that a dominant-class-first hierarchy actually lowers macro‑F1, sentiment features have minimal impact, and class‑weighted training yields the largest performance gains. Cross‑dataset transfer and slur reuse reveal that in‑domain metrics overstate real‑world reliability.
By Faria Naznin, Md Touhidur Rahman, Shahran Rahman Alve
arXiv:2610.33021v1 Announce Type: cross
Abstract: Large language models (LLMs) increasingly deliver information through personas, but it is unclear whether correcting the facts also makes their frami...
By Nouar AlDahoul, Myles Joshua Toledo Tan, Yasir Zaki
The paper introduces a diagnostic tool for distinguishing the use of misogynistic slurs from their mention in counter‑speech within code‑mixed Hinglish. It identifies evaluation artifacts in existing corpora, releases a 416‑item minimal‑pair contrast set that decorrelates slur presence and gendered register from labels, and proposes a pair‑consistency metric to assess model performance. Experiments show that even strong baselines struggle to consistently label counter‑speech pairs, while a large language model achieves perfect scores, indicating the benchmark measures genuine capability rather than exploitation of artifacts.
By Ashanvi Yadav, Shubham Bhardwaj
arXiv:2606. 28772v1 Announce Type: cross Abstract: Hate speech annotation pipelines routinely collapse annotator disagreement into majority vote labels before training.
By Joshua Muhumuza, Joab Ezra Agaba, Mercy Amiyo
arXiv:2502. 08266v3 Announce Type: replace-cross Abstract: Hate speech detection is a crucial task, especially on social media where harmful content can spread quickly.
By Somaiyeh Dehghan, Mehmet Umut Sen, Berrin Yanikoglu
arXiv:2609.00222v1 Announce Type: new
Abstract: Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how...
By Daniela Occhipinti, Andrea Piergentili, Marco Guerini
The study demonstrates that large language models (LLMs) can produce highly reliable labels under a single experimental setup, yet their outputs vary significantly when researchers alter task designs or model choices. By evaluating seven LLMs across 12 task designs and 3,000 tweets for offensive language and hate speech, the authors found that agreement dropped from a median Fleiss' κ of 0.91 to a median Cohen's κ of 0.76 when task designs changed. This design sensitivity inflates prevalence estimates by up to 110.6 times compared to sampling variance alone, and confidence scores fail to mitigate the issue.
By Thomas Reiter, Christoph Kern, Fedor Miasnikov, Sofiia Nikolenko, Rob Chew, Stephanie Eckman, Frauke Kreuter
The paper evaluates six structured decision models for hate‑speech moderation across four datasets, comparing them to specialized, zero‑shot, commercial, and supervised baselines. It investigates whether providing dataset definitions or decomposing them into multiple questions improves classification, and measures latency and cost. Results show that commercial LLMs outperform decision models on only one dataset, definitions alter up to 28% of predictions without consistent gains, and decomposition helps in only 20% of cases, yet the best hosted decision model approaches the best commercial LLM’s performance with roughly 97% lower inference cost.
By Demetris Paschalides, George Pallis, Marios D. Dikaiakos
Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how accurate a judge is, but whose judgments it repro...
HierSentNet is a hierarchical sentiment analysis framework designed to improve hate‑speech detection by combining a dominant‑class‑first hierarchy with sentiment features. In controlled experiments on Davidson et al. and HateXplain datasets, the hierarchy reduced macro‑F1 scores slightly, while sentiment features had minimal impact; however, class weighting significantly boosted macro‑F1 and hate‑specific F1. The framework also showed limited benefits on the OLID dataset and revealed robustness issues such as poor cross‑dataset transfer and high false‑positive rates for reclaimed slurs.
By Shahran Rahman Alve, Faria Naznin, Md Touhidur Rahman, Mohammad Ashrafuzzaman Khan
arXiv:2609.38036v2 Announce Type: replace-cross
Abstract: Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support to...
By Edoardo Bolzoni, Valerio Capraro
arXiv:2609.38036v1 Announce Type: cross
Abstract: Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with...
By Edoardo Bolzoni, Valerio Capraro