arXiv Computation and Language

Don' t Box Me In: Dynamic Cultural Adaptation and Cognitive Tracking for Social Understanding

arXiv Computation and Language
Oct 16

LLMs Infer Cultural Context but Fail to Apply It When Responding

The paper introduces CAPRI, a dataset of conversations designed to test whether large language models (LLMs) can adapt their responses to users’ cultural backgrounds. Experiments show that while LLMs can infer a user’s cultural context and recall relevant conventions, they frequently fail to apply this knowledge when generating answers unless explicitly prompted to do so. The study also finds that LLMs’ cultural adaptations improve as more cues are provided, yet their default priors are not culture‑neutral and sometimes reflect the model’s country of origin.

By Yisong Miao, Jian Zhu, Vered Shwartz
arXiv AI
Sep 30

Population Fidelity: Evaluating Population Representativeness in LLMs

The paper introduces Population Fidelity, an evaluation framework for assessing how well large language models (LLMs) represent human population attitudes. It focuses on three dimensions: group-level accuracy, between-group variation, and the structure of that variation. Using the framework, the authors replicate a prior study on machine bias and test cultural fine-tuning, finding that while fine-tuning improves overall alignment, it does not enhance representation of within-population differences.

By Neemias B. da Silva, Martin Lukk, Ali Sutani, Abhishek Moturu, Harris Yang, Daniel Silver, Matt Ratto, Thiago H. Silva
arXiv Computation and Language
Sep 21

Cultural Alignment in Large Language Models Using Soft Prompt Tuning

The paper proposes a method for culturally aligning large language models (LLMs) using soft prompt tuning optimized via Differential Evolution (DE). Unlike traditional fine‑tuning or reinforcement learning, this approach keeps model weights frozen and requires no preference data, instead leveraging aggregated survey scores from Hofstede's Value Survey Module (VSM13). Experiments on four countries and four instruction‑tuned models show that DE‑optimized prompts reduce cultural discrepancy, improve agreement with the World Values Survey, and are preferred in blinded pairwise evaluations by LLM judges.

By Reem I. Masoud, Martin Ferianc, Philip Treleaven, Miguel Rodrigues
arXiv AI
Jun 9

Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventions

arXiv:2606. 08081v1 Announce Type: cross Abstract: Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific conventions grounded in shared interaction history.

By Po-Ya Angela Wang, Chinmaya Mishra, Asl{\i} \"Ozy\"urek, Paula Rubio-Fern\'andez, Esam Ghaleb