arXiv AI
Jul 31
DialectLLM: A Dialect-Aware Dialog[ue] Generation Framework Beyond Standard American English
arXiv:2601. 22888v4 Announce Type: replace-cross Abstract: More than 80% of the 1.
EDRAC: Benchmarking Arabic Dialect Reading Comprehension
EDRAC is the first large‑scale benchmark for dialectal Arabic machine reading comprehension and generative question answering, covering five major dialects—Egyptian, Moroccan, Emirati, Syrian, and Saudi. It contains 499 passages from naturally spoken interactions and 4,977 QA pairs produced via a human–LLM collaborative pipeline. The benchmark evaluates Arabic‑centric and multilingual large language models, revealing gaps between semantic answer quality and dialectal fidelity and underscoring limitations of current evaluation metrics for dialectal Arabic generation.
Related stories
arXiv Computation and Language
Sep 11
Nuha-Speech: Building General-Purpose Arabic Speech-LLMs
Nuha‑Speech is a new initiative aimed at creating general‑purpose Arabic speech‑large language models (speech‑LLMs). It includes the construction of a large Arabic Speech Question‑Answering corpus with over 1.5 million samples for instruction tuning, supervised fine‑tuning of Qwen‑Omni model variants at various scales, and a systematic evaluation framework with diverse tasks and tailored metrics. The project seeks to establish foundational infrastructure for Arabic speech‑LLMs amid limited Arabic speech resources.
arXiv AI
Oct 1
WASIL: In-the-Wild Arabic Spoken Interactions with LLMs
arXiv:2605.16364v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) voice assistants are commonly built as cascaded Automatic Speech recognition (ASR) to LLM systems, where recogni...
arXiv Computation and Language
Sep 22
AlexandriaX 2026: The First Shared Task on Dialectal Arabic Machine Translation
arXiv:2609.22796v1 Announce Type: new Abstract: Dialectal Arabic machine translation (MT) remains challenging despite recent progress in Arabic language technologies, particularly because effective t...
Hugging Face Trending Papers
Jul 22
HalluTruthQA: A Fine-Grained Benchmark for Hallucination Detection, Localization, and Explanation in Arabic Question Answering
Large language models (LLMs) can generate fluent Arabic answers, yet factual errors remain difficult to detect, localize, explain, and verify. Existing hallucination benchmarks often provide response-level labels, with limited support for identifying the exact erroneous content, explaining why it is incorrect, or selecting the correct factual answer.
arXiv Machine Learning
Sep 11
E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets
E-CONAN introduces Arabic textual entailment and natural inference benchmarks comprising two datasets: E-CONAN-2 (2-way RTE) and E-CONAN-3 (3-way NLI). The datasets are built from automatically-translated pairs, human-validated machine translations, hand-crafted pairs from Arabic teaching books, and rumor-containing news headlines. The authors evaluated nine multilingual pretrained models and five large language models on these benchmarks, demonstrating that E-CONAN offers a more diverse and robust assessment than existing datasets like XNLI and ArNLI.
arXiv AI
Sep 1
Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models
The paper introduces a rubric-based benchmark to evaluate Saudi Arabic dialect and cultural competence in large language models. It comprises 31 expert-authored prompts covering idiomatic, pragmatic, lexical, and culturally embedded aspects, each paired with an expert-established ground truth. Four state-of-the-art models were scored, revealing that none exceeded 55% accuracy and that ambiguous framing was the most common error type.
arXiv Computation and Language
Sep 22
AskQE: Question Answering as Automatic Evaluation for Machine Translation
arXiv:2504.11582v3 Announce Type: replace Abstract: How can a monolingual English speaker determine whether an automatic translation in French is good enough to be shared? Existing MT error detection...
arXiv Computation and Language
Sep 24
NADI 2026: The Second Multidialectal Arabic Speech Processing Shared Task
NADI 2026 is the seventh edition of the Nuanced Arabic Dialect Identification shared task series and the second focused on multidialectal Arabic speech processing. It includes five main tasks—Automatic Speech Recognition, Spoken Dialect Identification, Text-to-Speech, Spoken Language Translation, and Spoken Language Understanding—along with eight subtasks that test realistic scenarios such as low‑bandwidth, mixed dialects, code‑switching, out‑of‑domain, and zero‑shot settings. The event attracted 21 teams from at least 13 countries, with 48 test‑phase submissions and 14 system‑description papers, and the results highlight out‑of‑domain generalization as a major bottleneck while showcasing the strengths of Arabic‑specialized speech models, multimodal dialect identification, and ensemble methods.
arXiv Computation and Language
Sep 2
Unified Multi-Dialectal Neural Machine Translation for Bangla Using the Dwadash Benchmark Corpus
arXiv:2608.12018v2 Announce Type: replace Abstract: Neural Machine Translation (NMT) and Large Language Models (LLMs) excel at cross-lingual tasks but often fail to capture intra-lingual morphologica...
arXiv Computation and Language
Sep 22
Some Dialects Are More Equal Than Others: Non-Prestigious Arabic Dialectal Bias in LLMs
arXiv:2609.23955v1 Announce Type: new Abstract: Previous work on Egyptian Arabic in NLP has focused largely on the prestigious Cairene Egyptian Arabic (CEA) dialect, resulting in a lack of representa...
arXiv Machine Learning
6d ago
AQL-Lite: Enabling Efficient Knowledge Acquisition for Arabic LLMs via Data-Centric Continual Pretraining
AQL‑Lite is a framework that enables efficient adaptation of multilingual large language models to Modern Standard Arabic by combining a quality‑first data curation pipeline with a partial‑parameter continual pretraining recipe. The approach trains RS‑LoRA adapters, the last eight transformer blocks, and the embedding matrix using layer‑wise learning rates, achieving significant benchmark gains while reducing training time and resource usage. In large‑scale experiments, AQL‑Lite recovers most of the performance of full‑parameter continual pretraining and outperforms adapter‑only baselines, demonstrating its practicality for Arabic LLM development.