arXiv:2604. 02951v2 Announce Type: replace-cross Abstract: Human data annotation, especially when involving experts, is often treated as an objective reference.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2606. 02255v1 Announce Type: cross Abstract: Human annotation is the empirical foundation of much NLP research, from dataset construction to model evaluation, but papers often leave unclear who produced the annotations and how the annotation process was controlled.
The paper investigates which demographic attributes large language models (LLMs) default to when annotating text without explicit demographic cues. By comparing non‑demographic, placebo‑conditioned, and demographic‑conditioned prompts on politeness and offensiveness tasks in the POPQUORN dataset, the authors find that LLMs exhibit notable gender, race, and age influences in their annotations. This contrasts with earlier studies that reported no such effects, highlighting the importance of considering demographic bias in LLM‑based annotation workflows.
The paper introduces AXLE, a pre‑annotation stage that uses a scenario vignette and guided conversation to generate a first‑person micro‑narrative for annotators. In a 2x2 study with 104 annotators on ambiguous hate‑speech items, AXLE increased identity resonance, sensitivity to harm, label entropy, and reduced inter‑annotator agreement, suggesting it surfaces plural perspectives rather than forcing consensus.
arXiv:2603. 13891v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for automated text annotation in tasks ranging from academic research to content moderation and hiring.
arXiv:2505.21710v2 Announce Type: replace Abstract: This study evaluates the effectiveness of ChatGPT, an advanced AI model for natural language processing, in identifying targeting and inappropriate...
arXiv:2609.24574v1 Announce Type: new Abstract: Computational social science increasingly relies on large language models for text annotation, and the validity of published findings now rests on the...