← Back to all news
arXiv Computation and Language By Yining Li, Huayang Li, Lemao Liu, Taro Watanabe

On the Position Bias of Large Language Models: A Comprehensive Study

Read the original on arXiv Computation and Language →

The paper investigates position bias in large language models (LLMs) across a wide range of model sizes—from billions to trillions of parameters—and across five languages. It finds that while small-scale models and non‑English languages still exhibit noticeable position bias, larger models and the use of Chain‑of‑Thought prompting substantially reduce this bias. Additionally, the study shows that emerging thinking models with advanced reasoning capabilities are almost immune to position bias, indicating a promising but resource‑intensive path to mitigating this vulnerability.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
6d ago

The Reliance Hypothesis: Chain-of-Thought Faithfulness Rises as Language Resource Level Falls

The paper investigates how Chain-of-Thought (CoT) faithfulness varies with language resource level and task type. Using MMMLU multiple-choice questions in eight languages, it finds that CoT faithfulness increases as resource level decreases for knowledge-intensive tasks but not for reasoning-intensive ones. The authors propose the Reliance Hypothesis, suggesting that models rely more on reasoning when they lack recall, especially in lower-resource languages, and note that model confidence partly explains this effect.

By Lucas Resck, Jingyi Sun, Qianli Wang, Isabelle Augenstein, Anna Korhonen
llms
More like this →
arXiv Computation and Language
Sep 3

Bridging Latent Reasoning and Target-Language Generation via Retrieval-Transition Heads

The paper investigates attention heads in multilingual Transformer models, distinguishing between retrieval heads that pull information from context and a newly identified class called Retrieval‑Transition heads (RTH) that direct the model toward a specific target language. Experiments across four multilingual benchmarks and two model families show that masking RTHs causes a larger performance drop than masking retrieval heads, indicating RTHs are crucial for chain‑of‑thought reasoning in multilingual LLMs. The study thus clarifies which attention heads are responsible for mapping to target languages, advancing our understanding of multilingual language models.

By Shaswat Patel, Vishvesh Trivedi, Yue Han, Yihuai Hong, Eunsol Choi
llmsbenchmarks
More like this →
arXiv Computation and Language
Sep 10

Deep and shallow biases in language models

The paper introduces a bias depth score to differentiate between stable model preferences (Deep biases) and prompt‑dependent responses (Shallow biases) in large language models. By analyzing 4,442 opinion prompts across four models, it finds that only about a quarter of concentrated preferences persist after scenario reframing, indicating that most are shallow. The study shows Deep biases are more often inherited from pretraining and harder to remove through fine‑tuning or prompt‑based debiasing, highlighting the need to distinguish learned biases from prompt artifacts.

By An Vo, Vy Tuong Dang, Khai-Nguyen Nguyen, Emilio Villa-Cueva, Thamar Solorio, Anh Totti Nguyen, Daeyoung Kim
llmsfine-tuningsafety
More like this →