arXiv Machine Learning July 31, 2026 By Agamdeep Singh, Ashish Tiwari, Hosein Hasanbeig, Priyanshu Gupta

Distributions In, Distributions Out: The Case for Soft-Label Training

Read the original on arXiv Machine Learning →

arXiv:2511. 14117v2 Announce Type: replace Abstract: Supervised classifiers output a distribution over classes but are typically trained against a single label obtained by collapsing multiple annotators into a majority vote.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 29

Learning from Annotation Uncertainty: Entropy-Aware Curriculum for Speech Emotion Recognition

arXiv:2606. 27536v1 Announce Type: cross Abstract: Speech emotion recognition (SER) often relies on hard consensus labels that collapse annotator disagreement.

By Zahra Omidi, John H. L. Hansen

safety

arXiv AI

Jun 30

Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain

arXiv:2606. 28772v1 Announce Type: cross Abstract: Hate speech annotation pipelines routinely collapse annotator disagreement into majority vote labels before training.

By Joshua Muhumuza, Joab Ezra Agaba, Mercy Amiyo

arXiv Computation and Language

Sep 2

Post-hoc Alignment of LLM-judges to Human Judgment Distribution

The paper introduces NAPHA, a lightweight post‑hoc alignment method that improves large language model (LLM) predictions of human judgment distributions (HJD) by matching LLM output distributions to HJD through entropy‑based class assignment and specialized alignment models. Experiments on five datasets show that while LLMs perform near human‑level on hard‑label tasks, they struggle with soft‑label predictions, and NAPHA consistently enhances soft‑label accuracy, especially on high‑entropy instances. The study also demonstrates that better entropy class prediction can further boost NAPHA’s effectiveness.

By Sebastian Steindl, Nikos Voskarides, Alberto Gasparin, Diego Marcheggiani

llms safety

arXiv Machine Learning

Jun 3

Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension

arXiv:2405. 03386v2 Announce Type: replace Abstract: Training with noisy class labels impairs neural networks' generalization performance.

By Marek Herde, Lukas L\"uhrs, Denis Huseljic, Bernhard Sick

benchmarks

arXiv AI

Sep 1

Do We Still Need Humans in the Loop? Human vs. LLM Annotation in Active Learning for TikTok Hate Speech Detection

arXiv:2604.13899v5 Announce Type: replace-cross Abstract: Annotating data remains a costly bottleneck for supervised NLP. Active learning (AL) reduces the number of human labels needed by selecting o...

By Ahmad Dawar Hakimi, Lea Hirlimann, Isabelle Augenstein, Hinrich Sch\"utze

llms nlp

arXiv Machine Learning

Jul 16

Temperature Scaling Is Not Enough: Calibration Gaps Under Human Label Distributions

arXiv:2607. 13423v1 Announce Type: new Abstract: Temperature scaling is the dominant post-hoc calibration method in modern deep learning.

By Wisdom Dogah

safety

One email a morning, machine-written

Related stories

Learning from Annotation Uncertainty: Entropy-Aware Curriculum for Speech Emotion Recognition

Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain

Post-hoc Alignment of LLM-judges to Human Judgment Distribution

Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension

Do We Still Need Humans in the Loop? Human vs. LLM Annotation in Active Learning for TikTok Hate Speech Detection

Temperature Scaling Is Not Enough: Calibration Gaps Under Human Label Distributions