Hugging Face Trending Papers
Rethinking the Evaluation and Optimization of LLM-Based Social Simulation
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Aug 21
Rethinking the Evaluation and Optimization of LLM-Based Social Simulation
arXiv:2608. 19689v1 Announce Type: new Abstract: LLM-based social simulation is a promising complement to traditional methods such as surveys and behavioral experiments.
arXiv Computation and Language
Sep 2
Post-hoc Alignment of LLM-judges to Human Judgment Distribution
The paper introduces NAPHA, a lightweight post‑hoc alignment method that improves large language model (LLM) predictions of human judgment distributions (HJD) by matching LLM output distributions to HJD through entropy‑based class assignment and specialized alignment models. Experiments on five datasets show that while LLMs perform near human‑level on hard‑label tasks, they struggle with soft‑label predictions, and NAPHA consistently enhances soft‑label accuracy, especially on high‑entropy instances. The study also demonstrates that better entropy class prediction can further boost NAPHA’s effectiveness.
arXiv Machine Learning
Jul 31
Distributions In, Distributions Out: The Case for Soft-Label Training
arXiv:2511. 14117v2 Announce Type: replace Abstract: Supervised classifiers output a distribution over classes but are typically trained against a single label obtained by collapsing multiple annotators into a majority vote.
arXiv Machine Learning
Jul 10
Ensemble Diversity Optimization for Subjective Supervision
arXiv:2607. 08493v1 Announce Type: new Abstract: Subjective NLP tasks often exhibit systematic annotator disagreement, requiring models that represent uncertainty rather than collapse it.
arXiv AI
Jun 15
OdysSim: Building Foundation Models for Human Behavior Simulation
arXiv:2606. 14199v1 Announce Type: cross Abstract: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation.
arXiv Machine Learning
Jun 5
SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs
arXiv:2606. 05376v1 Announce Type: new Abstract: Many human-centered tasks, including natural language inference (NLI) and emotion recognition (ER), have multiple plausible interpretations, leading to label ambiguity and challenging disagreements across human annotators.