YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper evaluates large language models (LLMs) on Arabic morphosyntactic tagging and dependency parsing, a challenging task due to rich morphology and orthographic ambiguity. It compares zero‑shot prompting with retrieval‑based in‑context learning across pre‑tokenized, raw‑text, and cascaded settings, finding that relevant demonstrations significantly boost performance. The best LLMs nearly match supervised systems but need extensive annotated data for demonstrations and high computational resources. All code and data are publicly released.
Nuha‑Speech is a new initiative aimed at creating general‑purpose Arabic speech‑large language models (speech‑LLMs). It includes the construction of a large Arabic Speech Question‑Answering corpus with over 1.5 million samples for instruction tuning, supervised fine‑tuning of Qwen‑Omni model variants at various scales, and a systematic evaluation framework with diverse tasks and tailored metrics. The project seeks to establish foundational infrastructure for Arabic speech‑LLMs amid limited Arabic speech resources.
arXiv:2608. 13580v1 Announce Type: cross Abstract: Jais 2 is a family of Arabic-centric large language models developed jointly by MBZUAI, Cerebras, and Inception, designed to advance Arabic-centric language modeling, with strong performance across the Arabic and culturally grounded benchmarks evaluated in this report.
The study evaluates Arabic–Russian machine translation by comparing seven fine‑tuned neural machine translation (NMT) models with four few‑shot large language models (LLMs) on a new 15.47 million‑pair corpus split into 20k/5k/5k. Fine‑tuned NLLB‑1.3B achieves the best performance (BLEU 16.3, COMET 0.738), while the best few‑shot LLM, Aya‑Expanse 8B, scores only BLEU 1.7 on 500 sentences. Error analysis shows that low lexical overlap between Arabic and Russian is the main source of failures, and statistical tests confirm significant performance gaps between most models.
arXiv:2609.06634v1 Announce Type: cross Abstract: LLMs' performance on machine translation (MT) tasks is often dependent on the data availability in the specific domains and language pairs that they...
The paper introduces AraSEG, a new Arabic sentence segmentation corpus covering eight genres and diverse punctuation and document structures. Experiments using AraSEG evaluate large language models, lightweight encoders, and dependency parser-based models, revealing that lightweight encoders and parser-based models outperform LLMs under the most challenging conditions. The study also shows that increasing training data size and genre diversity eventually saturates performance, that cross‑genre generalization remains difficult, and that accurate sentence segmentation significantly improves downstream dependency parsing.