The paper "Ghaib in Translation" investigates how large language models (LLMs) handle Urdu, a widely spoken language that is largely absent from safety evaluations. Five prominent LLMs—GPT‑4o, Claude Sonnet 4.5, Gemini 2.5 Flash, Qwen‑2.5, and Llama‑3.1—were tested on six datasets covering Nastaliq Urdu, Roman Urdu, English, and code‑switched Urdu‑English. The study found significant label instability between original‑script and English‑translation classifications, with missed‑in‑Urdu rates ranging from 2.4% to 9.9% (median 4.3%). A review of 205 papers across nine ALW/WOAH editions revealed no dedicated Urdu research, underscoring the language’s neglect in current safety research.
By Fawzia Zehra (Fuzzy), Kara-Isitt, Sonal Khosla, Stephen Swift
The paper evaluates large language models for hate‑speech detection in Roman Urdu, a low‑resource language with informal spelling variations. Using the Parameter‑Efficient Fine‑Tuning technique Low‑Rank Adaptation (LoRA), the authors fine‑tune models such as Mistral, LLaMA, Falcon, and multilingual BERT on the 72,000‑comment PURUTT dataset. While zero‑shot performance yields an F1 of 0.56, fine‑tuning a small fraction of parameters boosts F1 scores above 0.93, demonstrating that PEFT offers both high accuracy and computational efficiency for low‑resource language tasks.
By Toneema Zubair, Muhammad Junaid Asif, Faisal Kamiran, Hafiz Hassan Saeed, Rana Fayyaz Ahmad
The paper introduces UrduFactBench and UrduFactQA, two hand‑annotated benchmarks for claim verification and factual consistency evaluation in Urdu, created through a multi‑stage annotation process with native speakers. It also presents UrduFactCheck, a modular fact‑checking framework that uses both monolingual and translation‑based evidence retrieval to address the scarcity of high‑quality Urdu evidence. Experiments on twelve LLMs show that translation‑augmented pipelines outperform monolingual ones, highlighting ongoing challenges for open‑source models in Urdu.
By Sarfraz Ahmad, Hasan Iqbal, Momina Ahsan, Numaan Naeem, Muhammad Ahsan Riaz Khan, Arham Riaz, Muhammad Arslan Manzoor, Yuxia Wang, Preslav Nakov
arXiv:2608.21408v1 Announce Type: new
Abstract: Due to the widespread accessibility of the internet and social media, toxic and hateful con-tent has grown exponentially, causing significant distress...
By Toneema Zubair
The paper introduces BenCoDiC, a large Bengali cyberbullying corpus focused on political news discourse. It contains 50,250 textual instances from 100 YouTube videos, preserving caption–comment–reply relationships and organized via a nine‑dimensional star schema. A two‑tier human annotation process achieved high reliability (Fleiss' κ = 0.86, Krippendorff's α = 0.86), and transformer experiments show that adding conversational context modestly improves classification over isolated text.
By Md. Nesarul Hoque, Rudra Pratap Deb Nath, Umme Salma, Shima Chakraborty, Md. Hanif Seddiqui
arXiv:2608.10109v2 Announce Type: replace
Abstract: Social media has become a major venue for multilingual communication, where users frequently mix multiple languages within a single utterance. Alth...
By Ghazal Kalhor, Zahra Jafari, Amirarsalan Shahbazi, Behnam Bahrak
Multilingual Language Models like mBERT are widely used for low-resource NLP, yet their adaptation to morphologically inconsistent languages such as Roman Urdu remains underexplored. Roman Urdu spelling variation causes severe sub-word fragmentation, averaging 1.
arXiv:2610.20370v1 Announce Type: new
Abstract: Hate speech has become increasingly prevalent on social media, raising concerns about its detection, interpretation, and moderation. Persian, a relativ...
By Ghazal Kalhor, Sima Kamandloo, Mehrsa Shamsian, Behnam Bahrak
arXiv:2606.27314v2 Announce Type: replace
Abstract: To avoid moderation and surveillance on social media, some users routinely invent indirect linguistic expressions (ILE) that camouflage sensitive m...
By Hamid Reza Firoozfar, Mohammadsadegh Abolhasani, Reza Mousavi, Paul Jen-Hwa Hu
The paper introduces C$^2$TU, a training‑free and prompt‑free method for uncovering Chinese cloaked toxic content. It uses substring matching with a Chinese homophone graph and toxic lexicon to spot candidate toxic words, then filters non‑toxic candidates and corrects cloaks using two variants—one based on BERT and another on LLMs that overcome auto‑regressive limitations. Experiments on two Chinese toxic datasets show C$^2$TU outperforms the best competitor by up to 71% in F1 score and 35% in accuracy.
By Xuchen Ma, Jianxiang Yu, Wenming Shao, Bo Pang, Xiang Li
IndicSafeEval is a new evaluation framework that tests the safety robustness of large language models against persuasion-based jailbreak attacks in Indian languages. The benchmark includes 7,200 adversarial prompts covering ten safety-critical content categories, six persuasive strategies, and four languages (Hindi, Bengali, Marathi, Punjabi). Experiments show that model safety varies significantly across languages, prompt styles, and risk categories, revealing that current English-centric safety evaluations miss important multilingual vulnerabilities.
By Saikat Mondal, Mamta, Deeksha Varshney, Oana Cocarascu, Asif Ekbal
IndicSafeEval is a new evaluation framework that tests the safety robustness of large language models against persuasion-based jailbreak attacks in Indian languages. The benchmark covers ten safety-critical content categories, six persuasive strategies, and four languages—Hindi, Bengali, Marathi, and Punjabi—producing 7,200 adversarial prompts. Experiments show that model safety varies significantly across languages, prompt styles, and risk categories, highlighting gaps in current English-centric safety assessments.