The paper introduces CAPRI, a dataset of conversations designed to test whether large language models (LLMs) can adapt their responses to users’ cultural backgrounds. Experiments show that while LLMs can infer a user’s cultural context and recall relevant conventions, they frequently fail to apply this knowledge when generating answers unless explicitly prompted to do so. The study also finds that LLMs’ cultural adaptations improve as more cues are provided, yet their default priors are not culture‑neutral and sometimes reflect the model’s country of origin.
By Yisong Miao, Jian Zhu, Vered Shwartz
CuBEs introduces culturally‑situated behavioral evaluations for large language models, adding cultural context to test scenarios and assessments. The authors built a human‑labeled dataset covering 12 cultures, revealing significant cross‑cultural differences that one‑size‑fits‑all judgments miss. Evaluating 13 LLMs shows that culturally situated tests uncover varied behaviors, such as Western political bias versus non‑Western religious or colonial biases, which standard evaluations overlook.
By Hoda Ayad, Tanu Mitra, Abhishek Mukherji
arXiv:2608.28405v1 Announce Type: new
Abstract: Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common...
By Bryan Chen Zhengyu Tan, Weihua Zheng, Thong T. Doan, Bich Ngoc Doan, Jia Wang Peh, Xiaoyuan Yi, Jing Yao, Xing Xie, Nancy F. Chen, Zhengyuan Liu, JinYeong Bak, Wafi Shamdi, Soo Kai Chie, Liew Yu Siong, Aina Azyyati Binti Mohamad Rezal, Lew Yan Yan Vanessa, Huadan Wu, Dylan Raharja, Nadya Yuki Wangsajaya, Akane Fukushige, Kazushi Kato, Koji Inoue, Tatsuya Kawahara, Jaehyung Seo, Dongjun Kim, Seungyoon Lee, Zi Haur Pang, Rui Yang Tan, Charibeth Ko Cheng, Maria Regina Justina Estuar, Jann Railey Montalan, Pham Minh Duc, Roy Ka-Wei Lee
The study examines how large language models (LLMs) respond to simultaneous cultural identity cues, comparing explicit country names with implicit language cues. Using World Values Survey probing and open‑ended sociopolitical generation, it finds that language cues shift responses more than explicit cues but not toward the named country’s distribution, while explicit cues produce more differentiated value changes. Across 1.6 million responses, naming a country narrows the gap to the model’s own country‑specific answers by 30–44%.
By Siddhesh Pawar, Arnav Arora, Dustin Wright, Dirk Hovy, Isabelle Augenstein
The paper introduces Population Fidelity, an evaluation framework for assessing how well large language models (LLMs) represent human population attitudes. It focuses on three dimensions: group-level accuracy, between-group variation, and the structure of that variation. Using the framework, the authors replicate a prior study on machine bias and test cultural fine-tuning, finding that while fine-tuning improves overall alignment, it does not enhance representation of within-population differences.
By Neemias B. da Silva, Martin Lukk, Ali Sutani, Abhishek Moturu, Harris Yang, Daniel Silver, Matt Ratto, Thiago H. Silva
The paper investigates cultural biases in large language models (LLMs) by introducing the Culture-Related Open Questions (CROQ) dataset, which contains 24‑language questions about generic culture. Experiments reveal that LLMs disproportionately favor Japan in their responses, especially when prompted in high‑resource languages, while low‑resource languages tend to highlight countries where the language is official. The study also finds that these biases emerge after supervised fine‑tuning rather than during pre‑training.
By Joseba Fernandez de Landa, Carla Perez-Almendros, Jose Camacho-Collados
arXiv:2603. 16827v2 Announce Type: replace Abstract: Culture shapes reasoning, values, prioritization, and strategic decision-making, yet large language models (LLMs) often exhibit cultural biases that misalign with target populations.
By Maksim Eren, Eric Michalak, Brian Cook, Johnny Seales Jr
arXiv:2610.22545v1 Announce Type: new
Abstract: Stereotypes are subjective constructs that are mediated by the cultural context in which they are viewed. Successfully modeling them thus also requires...
By Maximilian Splieth\"over, Jannes L\"otfering, Dominik Bachmann, Alexander Espig, Willem Zuidema, Katrin Schulz, Henning Wachsmuth
arXiv:2605.12515v3 Announce Type: replace
Abstract: Multilingual large language models (MLLMs) often answer culturally ambiguous questions inconsistently across languages, even for a fixed user perso...
By Lucas Resck, Isabelle Augenstein, Anna Korhonen
The paper proposes a method for culturally aligning large language models (LLMs) using soft prompt tuning optimized via Differential Evolution (DE). Unlike traditional fine‑tuning or reinforcement learning, this approach keeps model weights frozen and requires no preference data, instead leveraging aggregated survey scores from Hofstede's Value Survey Module (VSM13). Experiments on four countries and four instruction‑tuned models show that DE‑optimized prompts reduce cultural discrepancy, improve agreement with the World Values Survey, and are preferred in blinded pairwise evaluations by LLM judges.
By Reem I. Masoud, Martin Ferianc, Philip Treleaven, Miguel Rodrigues
arXiv:2605.05662v2 Announce Type: replace-cross
Abstract: Current LLM safety benchmarks are predominantly English-centric and often rely on translation, failing to capture country-specific harms. Mor...
By Dasol Choi, Eugenia Kim, Jaewon Noh, Sang Seo, Eunmi Kim, Myunggyo Oh, Yunjin Park, Brigitta Jesica Kartono, Josef Pichlmeier, Helena Berndt, Sai Krishna Mendu, Glenn Johannes Tungka, \"Ozlem G\"ok\c{c}e, Suresh Gehlot, Katherine Pratt, Amanda Minnich, Haon Park
arXiv:2608.10503v2 Announce Type: replace
Abstract: As Large Language Models (LLMs) are increasingly deployed as autonomous agents, accurately evaluating their latent values and biases is critical. T...
By Davood Wadi, Mohsen Ghodrat, Matthew Philp