arXiv Computation and Language
Do Language Models Understand Igbo Culture? Beyond Multiple Choice
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Oct 19
Mitigating Cross-Lingual Cultural Inconsistencies in LLMs via Consensus-Driven Preference Optimisation
arXiv:2605.12515v3 Announce Type: replace Abstract: Multilingual large language models (MLLMs) often answer culturally ambiguous questions inconsistently across languages, even for a fixed user perso...
arXiv AI
Jun 8
The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
arXiv:2606. 07422v1 Announce Type: cross Abstract: Large language models are increasingly used to answer culturally grounded questions across languages, yet it remains unclear whether local cultural knowledge is better accessed through English or the local language.
arXiv Computation and Language
Oct 21
The Culture Within: Towards Mechanistic Understanding of Cultural Reasoning in LLMs
The paper introduces CultureLens, a benchmark with 9,261 scenarios from 75 countries that tests how large language models (LLMs) incorporate cultural information. Experiments on Llama‑3.1‑8B, Gemma‑3‑12B, and Qwen‑3.5‑27B show that presenting a contradictory cultural norm reduces aligned answers by 32‑47 percentage points, even when the model knows the correct country. Fine‑tuning with contradictory‑norm examples improves alignment to 85‑93% under contradictory norms while only slightly affecting performance under true norms.
arXiv Computation and Language
Sep 16
Beyond Cultural Knowledge: Evaluating Arabic Cultural Appropriateness of Large Language Models
arXiv:2609.16006v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly serve users whose expectations are shaped by their cultural context, yet most cultural evaluations test wha...
arXiv AI
Oct 5
CuBEs: Culturally-Situated Behavioral Evaluations and the Limitations of Culture-Blind LLM Judges
CuBEs introduces culturally‑situated behavioral evaluations for large language models, adding cultural context to test scenarios and assessments. The authors built a human‑labeled dataset covering 12 cultures, revealing significant cross‑cultural differences that one‑size‑fits‑all judgments miss. Evaluating 13 LLMs shows that culturally situated tests uncover varied behaviors, such as Western political bias versus non‑Western religious or colonial biases, which standard evaluations overlook.
arXiv Computation and Language
Oct 19
Cultural (In)Competence: Investigating Culture-Specific Stereotype Detection Capabilities of LLMs
arXiv:2610.22545v1 Announce Type: new Abstract: Stereotypes are subjective constructs that are mediated by the cultural context in which they are viewed. Successfully modeling them thus also requires...