arXiv AI
Sep 1

Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models

The paper introduces a rubric-based benchmark to evaluate Saudi Arabic dialect and cultural competence in large language models. It comprises 31 expert-authored prompts covering idiomatic, pragmatic, lexical, and culturally embedded aspects, each paired with an expert-established ground truth. Four state-of-the-art models were scored, revealing that none exceeded 55% accuracy and that ambiguous framing was the most common error type.

By Ghassan Al-Sumaidaee, Sajjad Abdoli, Ahmed Rashad, Maxim Legg
arXiv AI
Oct 1

TTLab at Daleel 2026: STAR-Ar, Sequence Tagging for Argument Recognition in Arabic

The paper introduces STAR‑Ar, a BERT‑BiLSTM‑CRF model designed for the Daleel 2026 Arabic argument mining shared task. It treats argument discourse unit detection and classification as a token‑level sequence labeling problem, achieving an F1‑score of 72.69 on validation and 73.7 on test data. Analysis shows that models trained only on editorial texts perform worse than those trained on debates, mainly due to the smaller editorial dataset.

By Bhuvanesh Verma, Ali Abusaleh, Alexander Mehler
arXiv Computation and Language
Aug 28

Why Current XAI Is Not Enough for Arabic NLP: A Critical Survey of the Explainability Gap

The paper surveys the state of Explainable AI (XAI) in Arabic NLP, highlighting three gaps: a method gap where Arabic XAI relies mainly on limited post‑hoc techniques; a task gap with most work focused on classification tasks and little on generation, retrieval, or dialogue; and a linguistic gap where explanations rarely address Arabic‑specific phenomena such as morphology, dialects, and diglossia. It proposes a taxonomy of tasks, methods, linguistic units, and evaluation practices, and outlines a research agenda for linguistically grounded Arabic XAI.

By Salima Lamsiyah, Ruslan Mitkov