arXiv Computation and Language

The Punjabi Geet Toxicity Dataset (PGT-FG): Fine-Grained Multimodal Annotations for Toxic Content in Punjabi Songs

arXiv AI
Aug 25

Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms

The paper introduces BanglaSafe, a benchmark of 879 Bengali prompts that covers 17 culturally grounded harm categories and five prompting conditions. Evaluation of 18 frontier LLMs shows that 53.6% of responses are unsafe or partially unsafe, with 14.7% containing strictly harmful content. The study finds that the writing style within Bengali has a stronger impact on safety than the language switch itself, and that current safety classifiers struggle to reliably evaluate Bengali content.

By Naymul Islam, Nusrat Jahan Lia, Shubhashis Roy Dipta, Sabik Bin Sultan, Abdullah Khan Zehady
arXiv Computation and Language
Oct 28

Lipi-Ghor: A Large-Scale, Diarized, Multi-Domain Bengali Speech Corpus and a Scalable Pipeline for Building It

Lipi-Ghor is a large-scale, weakly-supervised Bengali speech corpus containing 885 hours of audio from 998 YouTube videos across 17 domains, each annotated in SSTT format with speaker diarization and timestamps. It is the first Bengali corpus to combine extensive coverage with speaker-diarized, long-form audio annotations, and the authors provide a replicable pipeline for building similar resources in other languages. The dataset includes a human-annotated gold evaluation split and is released under CC‑BY‑4.0, with raw audio withheld.

By Sanjid Hasan, A H M Fuad, Risalat Labib, Bayazid Hasan Simon
arXiv Computation and Language
3d ago

BanglishDepYT: Depression Detection in Native Bengali Social Media with Emotion-Preserving Preprocessing and Domain-Adaptive Pretraining

BanglishDepYT is a native Bengali depression detection dataset comprising 191,000 unlabeled YouTube comments and 1,000 bilingual‑annotated samples. The authors introduce Emotion‑Preserving Hybrid preprocessing (EPH), which retains emoticons, Banglish tokens, negations, and emotional elongation, improving SVM F1 from 0.7062 to 0.8570. Domain‑adaptive pretraining of SahajBERT on in‑domain comments yields the best performance (F1 = 0.8161, MCC = 0.6272), and LLM‑assisted annotation enables pseudo‑labeling that further boosts AUC from 0.8499 to 0.8731.

By Ayon Roy, Sadman Sharif, Syed Shams Islam
arXiv AI
Aug 26

'Ghaib in Translation' aka Unseen Harm: Measuring Cross-Script Safety Inconsistency with 'Missed-in-Urdu' Scores in LLM Hate Speech Detection

The paper "Ghaib in Translation" investigates how large language models (LLMs) handle Urdu, a widely spoken language that is largely absent from safety evaluations. Five prominent LLMs—GPT‑4o, Claude Sonnet 4.5, Gemini 2.5 Flash, Qwen‑2.5, and Llama‑3.1—were tested on six datasets covering Nastaliq Urdu, Roman Urdu, English, and code‑switched Urdu‑English. The study found significant label instability between original‑script and English‑translation classifications, with missed‑in‑Urdu rates ranging from 2.4% to 9.9% (median 4.3%). A review of 205 papers across nine ALW/WOAH editions revealed no dedicated Urdu research, underscoring the language’s neglect in current safety research.

By Fawzia Zehra (Fuzzy), Kara-Isitt, Sonal Khosla, Stephen Swift
arXiv AI
Oct 27

Adapting LLM-SC to Living Sacred Heritage: Domain-Specific Extraction, Semantic Networks and Human Evaluation on Pashupatinath Temple Reviews

The paper adapts the LLM‑SC framework to extract structured information from visitor reviews of Pashupatinath Temple, Nepal, by adding eight sacred‑heritage entity types, four relation types, and nine aspect‑sentiment dimensions. Using GPT‑4o on 2,500 Tripadvisor reviews, the authors produced over 11,000 entity, 6,200 aspect‑sentiment, and 5,600 relation records, with the new sacred‑heritage types comprising 64.8% of entities. Human evaluation on a 200‑review subset yielded moderate precision and recall for entities and aspect‑sentiment extraction, and a 78.9% agreement on first‑listed relation candidates.

By Hari Sitaula, Manish Sakhakarmy, Pujan Pant, Sagar Newpane