arXiv AI By Constantinos Karouzos, Xingwei Tan, Nikolaos Aletras

An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift

Read the original on arXiv AI →

The paper investigates how preference tuning—optimizing language models with explicit preference signals—behaves when applied to new domains. It systematically compares five alignment objectives and several adaptation strategies, such as target‑domain supervised fine‑tuning and pseudo‑labeling, across summarization, question‑answering helpfulness, and safety tasks. Results show that while pseudo‑labeling reduces domain‑shift degradation, it also causes mode collapse, highlighting a trade‑off between generalization and diversity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 21

DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values

arXiv:2509. 08022v3 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) with diverse human values is essential for safe and effective deployment, yet existing benchmarks often overlook cultural and demographic variation.

By Yao Liang, Dongcheng Zhao, Feifei Zhao, Guobin Shen, Yuwei Wang, Dongqi Liang, Yi Zeng
arXiv Computation and Language
Sep 21

Cultural Alignment in Large Language Models Using Soft Prompt Tuning

The paper proposes a method for culturally aligning large language models (LLMs) using soft prompt tuning optimized via Differential Evolution (DE). Unlike traditional fine‑tuning or reinforcement learning, this approach keeps model weights frozen and requires no preference data, instead leveraging aggregated survey scores from Hofstede's Value Survey Module (VSM13). Experiments on four countries and four instruction‑tuned models show that DE‑optimized prompts reduce cultural discrepancy, improve agreement with the World Values Survey, and are preferred in blinded pairwise evaluations by LLM judges.

By Reem I. Masoud, Martin Ferianc, Philip Treleaven, Miguel Rodrigues