← Back to all news
arXiv Machine Learning By Florian Braun

Interpretable Cross-Lingual Alignment in Small Language Models: Probing Cultural and Pragmatic Reasoning in Japanese-English Bilingual LLMs

Read the original on arXiv Machine Learning →

arXiv:2608. 14896v1 Announce Type: cross Abstract: Large language models work well on English and behave in poorly understood ways on languages typologically far from it.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 5

M-GATE: Multilingual Grammar, Accuracy in Translation, and Efficiency Benchmark for Large Language Models

arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.

By Tom\'a\v{s} Burkert, Angelika Peljak-{\L}api\'nska, David Zelen\'y
llmsnlpbenchmarkssafety
More like this →
arXiv AI
Sep 3

VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages

VakyArth is the first pragmatic benchmark for Indic languages, covering Hindi, Punjabi, Tamil, and Malayalam. It tests models on five pragmatic phenomena—deixis, speech acts, implicature, social pragmatics, and coherence—using multiple-choice questions, natural language inference, and translation tasks authored by native speakers. Evaluation of multilingual LLMs shows consistent failures on pragmatic meanings rooted in Indic linguistic and cultural conventions, with systematic differences across languages and tasks.

By Usneek Singh, Poorvaja Veera Balaji Kumar, Parth Nanda, Anand Madhusoodanan, Geyang Guo, Wei Xu, Junyi Jessy L
llmsbenchmarks
More like this →