Redteaming Leading Arabic LLMs with ASAS
arXiv:2608.21985v1 Announce Type: new Abstract: As the adoption of large language models (LLMs) grows in Arabic-speaking regions, ensuring their safety and cultural alignment is increasingly critical...
arXiv:2608.21985v1 Announce Type: new Abstract: As the adoption of large language models (LLMs) grows in Arabic-speaking regions, ensuring their safety and cultural alignment is increasingly critical...
arXiv:2608.01291v2 Announce Type: replace Abstract: We present ArabicDialectSafety, a human-curated Arabic safety dataset of 25,071 prompts covering six Arabic varieties: Modern Standard Arabic, Syri...
arXiv:2605. 17173v2 Announce Type: replace-cross Abstract: Large language models exhibit safety degradation in non-English languages.
arXiv:2608. 14626v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved substantial progress in safety alignment, yet their safety guarantees remain significantly weaker in low-resource and multilingual settings than in high-resource languages.
Safety alignment in large language models (LLMs) is largely developed in English, assuming these safeguards generalize across multilingual settings. However, this assumption remains underexplored and exposes a vulnerability in low-resource languages.
arXiv:2605.05662v2 Announce Type: replace-cross Abstract: Current LLM safety benchmarks are predominantly English-centric and often rely on translation, failing to capture country-specific harms. Mor...
IndicSafeEval is a new evaluation framework that tests the safety robustness of large language models against persuasion-based jailbreak attacks in Indian languages. The benchmark includes 7,200 adversarial prompts covering ten safety-critical content categories, six persuasive strategies, and four languages (Hindi, Bengali, Marathi, Punjabi). Experiments show that model safety varies significantly across languages, prompt styles, and risk categories, revealing that current English-centric safety evaluations miss important multilingual vulnerabilities.
The paper introduces HausaSafeLM, a safety preference dataset for Hausa comprising 10,000 prompts (3,000 human‑corrected and 7,000 machine‑translated) across six harm categories. Using Qwen2.5‑7B‑Instruct, the authors compare supervised fine‑tuning (SFT) and SFT followed by Direct Preference Optimization (DPO) on 500 harmful and 200 benign Hausa prompts, finding that SFT improves refusal fluency and precision—especially with larger, noisier data—while DPO sharpens precision at the cost of linguistic quality. No single approach simultaneously maximizes safety, helpfulness, and fluency, and the study shows that refusal‑rate metrics alone are unreliable indicators of safety in low‑resource languages.
IndicSafeEval is a new evaluation framework that tests the safety robustness of large language models against persuasion-based jailbreak attacks in Indian languages. The benchmark covers ten safety-critical content categories, six persuasive strategies, and four languages—Hindi, Bengali, Marathi, and Punjabi—producing 7,200 adversarial prompts. Experiments show that model safety varies significantly across languages, prompt styles, and risk categories, highlighting gaps in current English-centric safety assessments.
The paper introduces MSafeTemp, a large multilingual safety alignment corpus covering 60 languages, created via a template-based augmentation pipeline that extracts safe templates from harmful prompts, translates them, and fills slots with culturally authentic entities. The authors demonstrate that evaluating safety solely in English can misjudge multilingual performance, as attacks that fail in English may succeed when translated, and culturally localized variants reveal additional failures. They also show that augmenting safety training data with these templates improves safety performance beyond English.
The paper introduces MMSAFE, a multi-layer framework designed to identify safety-degrading data in multilingual large language models. It shows that safety signals are distributed across multiple layers and only partially shared across languages, unlike the single-layer assumption used in monolingual settings. Experiments demonstrate that MMSAFE reduces harmful-response rates by 60% compared to random filtering and outperforms the best single-layer baseline across various models, languages, and safety benchmarks.
arXiv:2606. 08451v1 Announce Type: cross Abstract: Safety-aligned large language models often exhibit sycophancy, which is the tendency to affirm users' opinions regardless of factual accuracy.