arXiv AI
Measurement Validity in LLM Cultural Alignment
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
3d ago
LLMs Infer Cultural Context but Fail to Apply It When Responding
The paper introduces CAPRI, a dataset of conversations designed to test whether large language models (LLMs) can adapt their responses to users’ cultural backgrounds. Experiments show that while LLMs can infer a user’s cultural context and recall relevant conventions, they frequently fail to apply this knowledge when generating answers unless explicitly prompted to do so. The study also finds that LLMs’ cultural adaptations improve as more cues are provided, yet their default priors are not culture‑neutral and sometimes reflect the model’s country of origin.
arXiv AI
Jun 12
Occupational Prompting Reveals Cultural Bias in Large Language Models
arXiv:2606. 12443v1 Announce Type: cross Abstract: Social roles shape expectations, priorities, and judgments, yet it remains unclear how large language models (LLMs) associate occupational identities with broader cultural value patterns.
arXiv AI
Aug 10
People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe
arXiv:2608. 07367v1 Announce Type: new Abstract: As Large Language Models (LLMs) are increasingly used as a primary source of information and advice, understanding their alignment to humans in terms of values becomes a pressing concern.
arXiv Computation and Language
Oct 12
The Alignment Veto: How Safety Training Suppresses Cultural Knowledge in LLMs
The paper investigates how alignment training in large language models (LLMs) can suppress cultural knowledge rather than erase it. By surveying 1.53 million humans across 16 MENA countries and testing 26 models, the authors find that when models refuse to answer, their internal logit distributions still align closely with human responses, a phenomenon they term the "alignment veto." They distinguish between suppression failures—where accurate internal representations are blocked at output—and representational bias failures—where the encoding itself diverges from human values, and show that these require different interventions. The study also reveals significant inequities in the safety tax, with a 37.6 % cost and a 19.8 % alignment-quality gap between best- and worst-served nations, and notes that native-language prompting actually widens this gap. Sparse autoencoder analysis identifies a candidate feature in the DPO stage that mediates suppression in the Tulu-3-8B model.
arXiv AI
Sep 25
Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populations
The paper introduces Cultural Divergence Preservation (CDP), a new diagnostic for evaluating whether large language models (LLMs) preserve cross‑country differences when used as synthetic survey respondents. CDP uses a single human calibration to detect cultural flattening (reduced divergence) or caricature (increased divergence) and is shown to vary monotonically with cross‑country divergence, unlike conventional Jensen–Shannon divergence metrics. Experiments across multiple LLM backbones, prompting methods, and survey domains reveal that CDP uncovers systematic discrepancies with traditional fidelity metrics, highlighting that methods favored by those metrics can still produce strong flattening.
arXiv AI
Jul 29
Personalization, Personas, and Forecasting in Value Alignment
arXiv:2607. 24782v1 Announce Type: new Abstract: LLM behavior may be conditioned by human identity in several ways: they may be asked to adapt to users, role-play populations, or forecast how people would answer value-laden questions.