arXiv AI June 16, 2026 By Rafael Ferreira, In\^es Vieira, In\^es Calvo, James Furtado, Iago Paulo, Diogo Tavares, Diogo Gl\'oria-Silva, David Semedo, Jo\~ao Magalh\~aes

P3B3: A Multi-Turn Conversational Benchmark for Measuring European and Brazilian Portuguese Variety Bias in LLMs

Read the original on arXiv AI →

arXiv:2606. 16753v1 Announce Type: cross Abstract: As Large Language Models (LLMs) become embedded in everyday communication, capturing regional linguistic variation is essential for reliable and equitable language use.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Sep 10

Deep and shallow biases in language models

The paper introduces a bias depth score to differentiate between stable model preferences (Deep biases) and prompt‑dependent responses (Shallow biases) in large language models. By analyzing 4,442 opinion prompts across four models, it finds that only about a quarter of concentrated preferences persist after scenario reframing, indicating that most are shallow. The study shows Deep biases are more often inherited from pretraining and harder to remove through fine‑tuning or prompt‑based debiasing, highlighting the need to distinguish learned biases from prompt artifacts.

By An Vo, Vy Tuong Dang, Khai-Nguyen Nguyen, Emilio Villa-Cueva, Thamar Solorio, Anh Totti Nguyen, Daeyoung Kim

llms fine-tuning safety

arXiv Machine Learning

Jun 3

From Script to Semantics: Prompting Strategies for African NLI

arXiv:2606. 03304v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains underexplored especially under pure prompting without fine-tuning.

By Anuj Tiwari, Terry Oko-odion, Hannah Nwokocha

llms fine-tuning benchmarks

arXiv AI

Jul 9

DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation

arXiv:2607. 07669v1 Announce Type: cross Abstract: Large language models increasingly \emph{understand} dialectal English, yet still \emph{produce} only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed.

By Jordan Painter, Dipankar Srirag, Adarsh Kappiyath, Diptesh Kanojia, Aditya Joshi, Lu Yin

llms fine-tuning benchmarks safety

Hugging Face Trending Papers

Aug 6

Different Perturbations, Different Mechanisms: Understanding Continued Pre-training for Zero-Shot Dialect Robustness

Dialectal variation remains a major challenge for multilingual language models. Perturbation-based continued pre-training (CPT) has emerged as a promising approach to improving robustness, yet existing work largely evaluates individual perturbation strategies in isolation and provides limited insight into why they work.

llms safety

arXiv Computation and Language

Sep 22

Some Dialects Are More Equal Than Others: Non-Prestigious Arabic Dialectal Bias in LLMs

arXiv:2609.23955v1 Announce Type: new Abstract: Previous work on Egyptian Arabic in NLP has focused largely on the prestigious Cairene Egyptian Arabic (CEA) dialect, resulting in a lack of representa...

By Mai Mohamed Eida, Ryan Dolan, Paul de Nijs, Jonathan Dunn

llms nlp benchmarks safety

arXiv AI

Jun 24

On the Stability of Prompt Ranking in Large Language Model Evaluation

arXiv:2606. 24381v1 Announce Type: cross Abstract: Prompt-based interaction has become a dominant paradigm for using large language models (LLMs), where multiple candidate prompts are evaluated and the top-ranked one is selected for downstream use.

By Shaoshuai Du, Penghao Liang, Yixian Shen, Chuanqi Shi, Hang Zhang, Lun Wang

llms benchmarks