The paper investigates whether lower stereotype bias scores in multilingual language models truly indicate greater fairness. By auditing ten open models across eight languages, the authors show that lower average scores often arise from translation artifacts rather than genuine bias reduction, as item‑level correlations between English and translated scores are weak. They also identify pipeline errors that can mask bias signals and provide code to replicate their findings.
By Harsh Kumar
arXiv:2605.29637v2 Announce Type: replace
Abstract: Large language models often exhibit a substantial gap between their performance in English and in lower-resourced languages on equivalent knowledge...
By Debajyoti Mazumder, Divyansh Pathak, Prashant Kodali, Aditya Joshi, Akshay Agarwal, Jasabanta Patro
arXiv:2507.23248v2 Announce Type: replace-cross
Abstract: Bengali is spoken by more than 230 million people, yet no standardized instrument evaluates large language models (LLMs) on Bengali across th...
By Shimanto Bhowmik, Tawsif Tashwar Dipto, Md Sazzad Islam, Sheryl Hsu, Tahsin Reasat
The study replicates a distributional‑semantics extractive summarisation method for Hindi, adapting all language‑specific components to Devanagari. Evaluated on the Hindi portions of XL‑Sum and FIRE ILSUM 2.0 with a Devanagari‑aware ROUGE scorer, the replicated system performs significantly worse than a simple three‑sentence lead baseline. Feature ablation shows that sentence position alone reproduces the lead baseline, while other features only steer extraction toward long, entity‑dense body sentences, and TextRank performs identically.
"whyItMatters":"The results indicate that current Hindi summarisation benchmarks cannot reward non‑lead content selection, highlighting the need for purpose‑built evaluation resources."
By Showket Ahmad Khan, Mudasir Mohd, Nasrullah Sheikh, Mohsin Altaf Wani, Abid Hussain Wani, Hilal Ahmad Khanday, Niyaz Ahmad Wani
arXiv:2604. 04532v2 Announce Type: replace-cross Abstract: Evaluation language is typically treated as a fixed English default in agentic code benchmarks, yet we show that changing the judge's language can invert backbone rankings.
By Alhasan Mahmood, Samir Abdaljalil, Hasan Kurban
arXiv:2607. 24276v1 Announce Type: cross Abstract: Large language models (LLMs) process text through subword tokenizers rather than directly reading characters or words.
By Priyansh Srivastava