The Flow has not summarised this story yet — read it at arXiv AI.
The paper "Ghaib in Translation" investigates how large language models (LLMs) handle Urdu, a widely spoken language that is largely absent from safety evaluations. Five prominent LLMs—GPT‑4o, Claude Sonnet 4.5, Gemini 2.5 Flash, Qwen‑2.5, and Llama‑3.1—were tested on six datasets covering Nastaliq Urdu, Roman Urdu, English, and code‑switched Urdu‑English. The study found significant label instability between original‑script and English‑translation classifications, with missed‑in‑Urdu rates ranging from 2.4% to 9.9% (median 4.3%). A review of 205 papers across nine ALW/WOAH editions revealed no dedicated Urdu research, underscoring the language’s neglect in current safety research.
The paper evaluates large language models for hate‑speech detection in Roman Urdu, a low‑resource language with informal spelling variations. Using the Parameter‑Efficient Fine‑Tuning technique Low‑Rank Adaptation (LoRA), the authors fine‑tune models such as Mistral, LLaMA, Falcon, and multilingual BERT on the 72,000‑comment PURUTT dataset. While zero‑shot performance yields an F1 of 0.56, fine‑tuning a small fraction of parameters boosts F1 scores above 0.93, demonstrating that PEFT offers both high accuracy and computational efficiency for low‑resource language tasks.
The paper introduces UrduFactBench and UrduFactQA, two hand‑annotated benchmarks for claim verification and factual consistency evaluation in Urdu, created through a multi‑stage annotation process with native speakers. It also presents UrduFactCheck, a modular fact‑checking framework that uses both monolingual and translation‑based evidence retrieval to address the scarcity of high‑quality Urdu evidence. Experiments on twelve LLMs show that translation‑augmented pipelines outperform monolingual ones, highlighting ongoing challenges for open‑source models in Urdu.
arXiv:2608.21408v1 Announce Type: new Abstract: Due to the widespread accessibility of the internet and social media, toxic and hateful con-tent has grown exponentially, causing significant distress...
The paper introduces BenCoDiC, a large Bengali cyberbullying corpus focused on political news discourse. It contains 50,250 textual instances from 100 YouTube videos, preserving caption–comment–reply relationships and organized via a nine‑dimensional star schema. A two‑tier human annotation process achieved high reliability (Fleiss' κ = 0.86, Krippendorff's α = 0.86), and transformer experiments show that adding conversational context modestly improves classification over isolated text.
arXiv:2608.10109v2 Announce Type: replace Abstract: Social media has become a major venue for multilingual communication, where users frequently mix multiple languages within a single utterance. Alth...