arXiv Computation and Language
Sep 2

Post-hoc Alignment of LLM-judges to Human Judgment Distribution

The paper introduces NAPHA, a lightweight post‑hoc alignment method that improves large language model (LLM) predictions of human judgment distributions (HJD) by matching LLM output distributions to HJD through entropy‑based class assignment and specialized alignment models. Experiments on five datasets show that while LLMs perform near human‑level on hard‑label tasks, they struggle with soft‑label predictions, and NAPHA consistently enhances soft‑label accuracy, especially on high‑entropy instances. The study also demonstrates that better entropy class prediction can further boost NAPHA’s effectiveness.

By Sebastian Steindl, Nikos Voskarides, Alberto Gasparin, Diego Marcheggiani
arXiv Computation and Language
Oct 22

From Value Conditioning to Behavioral Shift: Lightweight Value Alignment of LLMs

The paper investigates whether fine‑tuning large language models (LLMs) on value survey responses can shift their value‑conditioned outputs and downstream behavior. Researchers first estimate baseline value profiles for several open‑source LLMs using the ten Schwartz values, then fine‑tune the models on survey‑style supervision that selectively alters target value ratings while preserving non‑target values. The resulting shifts are evaluated on held‑out survey questions and on out‑of‑domain behavioral proxies, such as a contextual moral‑judgment benchmark from Reddit AITA posts and a text‑based interactive decision‑making benchmark. The study finds that survey‑based supervision can change both survey responses and downstream behavior, but the transfer is strongly model‑ and value‑dependent and influenced by how value descriptions are framed.

By Shangrui Nie, Florian Mai, David Kacz\'er, Charles Welch, Zhixue Zhao, Lucie Flek