arXiv Computation and Language
When Similar Means Different: Evaluating LLMs on Arabic--Hebrew Cognates
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Sep 2
ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs
arXiv:2607.00171v2 Announce Type: replace Abstract: Text embeddings are standard for semantic similarity tasks, yet their evaluation remains an open challenge. Current benchmarks are static, cover on...
arXiv Machine Learning
1d ago
HeRE: A Novel Benchmark for Hebrew Relation Extraction via LLM-Guided Denoising of Knowledge Graph Alignments
arXiv:2610.22542v1 Announce Type: cross Abstract: While LLMs demonstrate strong capabilities in English Relation Extraction (RE) tasks, a significant gap remains in assessing their RE performance in...
arXiv Computation and Language
Aug 27
Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation
The paper investigates how to fairly compare language models across languages, noting that current evaluation methods vary widely and lack empirical validation. By training controlled monolingual models on parallel data and testing multilingual LLMs, the authors find that many normalized metrics suffer from biases due to tokenization, encoding, and orthographic differences. Instead, they recommend using sentence‑level negative log‑likelihood over semantically equivalent sequences for more reliable cross‑lingual comparisons.
arXiv AI
Sep 10
Mind the Gap: Exposing LLM Translation Blind Spots Using the AlphaMWE Multilingual Parallel Corpus
arXiv:2609.06634v1 Announce Type: cross Abstract: LLMs' performance on machine translation (MT) tasks is often dependent on the data availability in the specific domains and language pairs that they...
arXiv AI
Aug 7
MameLoshnLM: Yiddish Language Model and Evaluation Benchmark
arXiv:2608. 05850v1 Announce Type: cross Abstract: We present MameLoshnLM, the first open-source 8B-parameter language model built specifically for Yiddish.
arXiv AI
Aug 17
Jais 2: A Family of Arabic-Centric Open Large Language Models
arXiv:2608. 13580v1 Announce Type: cross Abstract: Jais 2 is a family of Arabic-centric large language models developed jointly by MBZUAI, Cerebras, and Inception, designed to advance Arabic-centric language modeling, with strong performance across the Arabic and culturally grounded benchmarks evaluated in this report.