← Back to all news
arXiv Computation and Language By Umaira Izhar, Gunjan Arora, Pushpendra Singh

Measuring Cultural Alignment Beyond the Average: A Framework for Evaluating Maternal-Health LLM Interactions in Indian Contexts

Read the original on arXiv Computation and Language →

The paper introduces MH‑INDIC, a framework that evaluates maternal‑health language model interactions in North Indian urban and semi‑urban settings by measuring cultural alignment across ten dimensions of reasoning. Using a survey of 102 pregnant and postpartum women, the authors assess ten LLMs, finding that while models approximate overall population‑level cultural patterns, they lack the demographic and household‑specific variation seen in humans. The study also demonstrates that human‑grounded conditioning improves both profile alignment and dialogue quality in generated interactions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
2d ago

Whose Values, Which Contexts, and What Signals? Grounding Cultural Value Alignment with Re3Culture

arXiv:2610.41793v1 Announce Type: new Abstract: Cultural value alignment requires not only model optimization but also a well-specified target grounded in real-human evidence. Yet how such targets sh...

By Jing Yao, Hyunjin Hwang, Weihua Zheng, Bryan Chen Zhengyu Tan, Jia Wang Peh, Dylan Raharja, JinYeong Bak, Roy Ka-Wei Lee, Rui Cao, Yuki Arase, Jacki O'Neill, Zhicheng Dou, Xing Xie, Xiaoyuan Yi, Furu Wei
llmssafety
More like this →
arXiv Computation and Language
Oct 16

LLMs Infer Cultural Context but Fail to Apply It When Responding

The paper introduces CAPRI, a dataset of conversations designed to test whether large language models (LLMs) can adapt their responses to users’ cultural backgrounds. Experiments show that while LLMs can infer a user’s cultural context and recall relevant conventions, they frequently fail to apply this knowledge when generating answers unless explicitly prompted to do so. The study also finds that LLMs’ cultural adaptations improve as more cues are provided, yet their default priors are not culture‑neutral and sometimes reflect the model’s country of origin.

By Yisong Miao, Jian Zhu, Vered Shwartz
llmsbenchmarks
More like this →
arXiv AI
Oct 5

CuBEs: Culturally-Situated Behavioral Evaluations and the Limitations of Culture-Blind LLM Judges

CuBEs introduces culturally‑situated behavioral evaluations for large language models, adding cultural context to test scenarios and assessments. The authors built a human‑labeled dataset covering 12 cultures, revealing significant cross‑cultural differences that one‑size‑fits‑all judgments miss. Evaluating 13 LLMs shows that culturally situated tests uncover varied behaviors, such as Western political bias versus non‑Western religious or colonial biases, which standard evaluations overlook.

By Hoda Ayad, Tanu Mitra, Abhishek Mukherji
llmssafety
More like this →