arXiv Computation and Language
Sep 1

When Hate Meets Facts: LLMs-in-the-Loop for Check-worthiness Detection in Hate Speech

The paper introduces WSF-ARG+, a new dataset that pairs hate speech with check‑worthiness annotations, and presents an LLM‑in‑the‑loop framework to streamline the annotation process. Experiments with 12 open‑weight large language models demonstrate that the framework cuts human effort while maintaining annotation quality. The study also shows that incorporating check‑worthiness labels improves hate‑speech detection performance, boosting macro‑F1 scores for large models by up to 0.213 and averaging 0.154 across models.

By Nicol\'as Benjam\'in Ocampo, Tommaso Caselli, Davide Ceolin