IndicTriMix presents a new benchmark and models for token‑level language identification in tri‑language code‑mixed text involving Hindi, Gujarati, and Bengali. The authors reformulate the task as sequence labeling and fine‑tune transformer models MuRIL and XLM‑RoBERTa, evaluating them on manually annotated test sets. They also introduce two code‑mixing generation methods using parallel sentences and release the datasets and fine‑tuned models for public use.
By Pruthwik Mishra, Rudra Trivedi, Avi Patel, Ashok Urlana, Shrikant Malviya
arXiv:2610.15795v1 Announce Type: cross
Abstract: Automated moderation systems are deployed at scale, yet they remain opaque to the users and moderators affected by their decisions, and their coverag...
By Muhammad Shahid Iqbal Malik
arXiv:2606. 24825v1 Announce Type: cross Abstract: Part-of-Speech (POS) tagging is a foundational NLP task underpinning machine translation, information extraction, and syntactic parsing.
By Hariom Ingle, Ronit Ghode, Ishwari Gondkar, Jidnyasa Harad, Raviraj Joshi
arXiv:2606.27314v2 Announce Type: replace
Abstract: To avoid moderation and surveillance on social media, some users routinely invent indirect linguistic expressions (ILE) that camouflage sensitive m...
By Hamid Reza Firoozfar, Mohammadsadegh Abolhasani, Reza Mousavi, Paul Jen-Hwa Hu
arXiv:2610.20370v1 Announce Type: new
Abstract: Hate speech has become increasingly prevalent on social media, raising concerns about its detection, interpretation, and moderation. Persian, a relativ...
By Ghazal Kalhor, Sima Kamandloo, Mehrsa Shamsian, Behnam Bahrak
arXiv:2610.20279v1 Announce Type: new
Abstract: Part-of-speech tagging is a fundamental sequence labeling task and an important component for many downstream natural language processing applications....
By Usman Nawaz, Muhammad Junaid Iqbal, Muhammad Asaf, Mariam Munsif Mir, Faisal Rehman, Tahir Alyas, Muhammad Hasnain
arXiv:2610.16601v1 Announce Type: cross
Abstract: Processing multilingual text requires both identifying a broad range of languages and distinguishing the language of each token in inputs containing...
By Seongtae Hong, Youngjoon Jang, Seungyoon Lee, Jungseob Lee, Heuiseok Lim
arXiv:2609.25298v1 Announce Type: new
Abstract: Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely...
By Yusser Al Ghussin, Eva Gavaller, Cristina Espa\~na-Bonet, Josef van Genabith, Simon Ostermann
arXiv:2609.16393v1 Announce Type: new
Abstract: We introduce ParsHate, a manually annotated dataset of 10,000 Persian tweets spanning 2013-2022, representing the first decade-long benchmark for hate...
By Zahra Bokaei, Walid Magdy, Bonnie Webber
arXiv:2607. 21255v1 Announce Type: cross Abstract: Slang is a central component of everyday language, reflecting linguistic creativity, social identity, and cultural change, yet its dy- namic and non-standard nature makes it difficult to model computationally.
By Panagiotis Papadakos, Katerina Papantoniou, Dimitris Plexousakis
arXiv:2606. 30790v1 Announce Type: cross Abstract: Romanized Code Mixing (RCM), where bilingual speakers fluidly blend local languages with English in Roman script, has emerged as the dominant form of communication across multilingual communities.
By Avisha Das, Mihir Parmar, Mohana Ramnath, Pulkit Verma
arXiv:2606. 20089v1 Announce Type: cross Abstract: Persian pretrained language models (PLMs) are still limited by the scarcity of large-scale, high-quality pretraining corpora and by insufficient evaluation beyond standard classification and NER tasks.
By Arash Ghafouri, Mahdi Firouzmandi, Hossein Saberi, Mohammad Reza Hasani Ahangar