arXiv:2511. 14117v2 Announce Type: replace Abstract: Supervised classifiers output a distribution over classes but are typically trained against a single label obtained by collapsing multiple annotators into a majority vote.
arXiv:2606. 27536v1 Announce Type: cross Abstract: Speech emotion recognition (SER) often relies on hard consensus labels that collapse annotator disagreement.
arXiv:2606. 28772v1 Announce Type: cross Abstract: Hate speech annotation pipelines routinely collapse annotator disagreement into majority vote labels before training.
The paper introduces NAPHA, a lightweight post‑hoc alignment method that improves large language model (LLM) predictions of human judgment distributions (HJD) by matching LLM output distributions to HJD through entropy‑based class assignment and specialized alignment models. Experiments on five datasets show that while LLMs perform near human‑level on hard‑label tasks, they struggle with soft‑label predictions, and NAPHA consistently enhances soft‑label accuracy, especially on high‑entropy instances. The study also demonstrates that better entropy class prediction can further boost NAPHA’s effectiveness.
arXiv:2405. 03386v2 Announce Type: replace Abstract: Training with noisy class labels impairs neural networks' generalization performance.
arXiv:2604.13899v5 Announce Type: replace-cross Abstract: Annotating data remains a costly bottleneck for supervised NLP. Active learning (AL) reduces the number of human labels needed by selecting o...
arXiv:2607. 13423v1 Announce Type: new Abstract: Temperature scaling is the dominant post-hoc calibration method in modern deep learning.
arXiv:2606. 14965v1 Announce Type: new Abstract: Synthetic instance-dependent label noise (IDN) benchmarks are widely used to evaluate noisy-label learning methods, yet existing approaches typically generate noise through imperfect annotators or classifier raters, leaving the source of ambiguity implicit.
arXiv:2606. 05376v1 Announce Type: new Abstract: Many human-centered tasks, including natural language inference (NLI) and emotion recognition (ER), have multiple plausible interpretations, leading to label ambiguity and challenging disagreements across human annotators.
arXiv:2608. 19689v1 Announce Type: new Abstract: LLM-based social simulation is a promising complement to traditional methods such as surveys and behavioral experiments.
arXiv:2607. 08493v1 Announce Type: new Abstract: Subjective NLP tasks often exhibit systematic annotator disagreement, requiring models that represent uncertainty rather than collapse it.
FedCC is a new algorithm for distillation-based federated learning that tackles label distribution skew by allowing clients to mark ambiguous samples as 'unknown' instead of forcing a potentially wrong classification. By adding this extra class and calibrating pseudo-labels on a public dataset, FedCC balances confidence across majority and minority classes. Experiments show that FedCC outperforms existing methods, achieving 67.3% accuracy even when each client has data from only one of ten classes, whereas baselines drop to near-random performance.
LLM-based social simulation is a promising complement to traditional methods such as surveys and behavioral experiments. A core question is how to evaluate the fidelity of LLM-simulated human behavior...