arXiv:2009. 10277v2 Announce Type: replace-cross Abstract: We propose a system for measuring hate speech on a continuous, interval-valued spectrum ranging from genocidal to supportive speech by combining supervised deep learning with faceted Rasch item response theory (IRT).
arXiv:2608. 04056v1 Announce Type: cross Abstract: When people label text for sexism, they often disagree, and not because some of them are wrong: they genuinely perceive sexism differently.
ValueGraph is a graph pre‑training framework that incorporates automatically inferred moral‑value signals as soft constraints to learn contextualized user representations. By constructing post‑reply graphs, it captures both semantic and structural information and aligns users through contrastive and clustering objectives based on relative value similarity. Experiments on stance detection and Twitter bot detection demonstrate consistent improvements over strong text‑based, graph‑based, and LLM baselines, underscoring the benefit of value‑signal guidance for socially informed user modeling.
Understanding moral values in social media text offers insight into moral judgement formation, and supervised NLP models trained on crowdsourced data have achieved strong classification performance. However, most approaches simplify the problem by aggregating multiple annotators' labels into a single "ground truth", overlooking the inherent subjectivity of the task.
arXiv:2502. 08266v3 Announce Type: replace-cross Abstract: Hate speech detection is a crucial task, especially on social media where harmful content can spread quickly.
arXiv:2607. 08493v1 Announce Type: new Abstract: Subjective NLP tasks often exhibit systematic annotator disagreement, requiring models that represent uncertainty rather than collapse it.
arXiv:2601. 02813v3 Announce Type: replace Abstract: Aligning language models to qualitative behavioral traits, such as human-likeness, remains difficult because they are hard to define, measure, and optimize.
arXiv:2602. 03160v2 Announce Type: replace Abstract: Aligning Large Language Models (LLMs) with the diverse spectrum of human values remains a central challenge: preference-based methods often fail to capture deeper motivational principles.
arXiv:2608.30842v1 Announce Type: new Abstract: Humans play a vital role at every stage of AI development, from data collection and curation to model development and evaluation. However, humans often...
arXiv:2403.17612v3 Announce Type: replace Abstract: Labeling corpora constitutes a bottleneck to create models for new tasks or domains. Large language models mitigate the issue with automatic corpus...
Large Language Models (LLMs) as judges across various scenarios such as assessing model responses is becoming an increasingly accepted paradigm. However, existing judgment approaches often rely on trained judgers using fixed preference data, which tend to overlook diverse user preferences and struggle to adapt to real-world human-AI dialogue scenarios.
MAVEN is a hierarchical framework for evaluating whether multimodal content aligns with macro‑societal values such as peace, justice, and freedom. It organizes values into six primary dimensions and 72 secondary indicators, enabling multi‑level quantitative scoring. The authors build a human‑verified multimodal benchmark, a soft‑match metric, and propose efficient evaluator optimization techniques, demonstrating that a compact 2B evaluator performs comparably to larger models and approaches state‑of‑the‑art closed‑source VLMs.