arXiv AI
Same Judgment, Different Yes: Measuring Multilingual Judgment Profiles in Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Jul 6
The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment
Large language models (LLMs) increasingly issue judgments read as binary verdicts, and a growing literature reports such judgments shifting under logically irrelevant changes of wording - among them an amplified yes-no bias on moral dilemmas, absent in humans. A single framing cannot say what such a shift is: in a yes/no question the word "no" is at once logical verdict, lexical token, and last-printed option.
arXiv AI
Jul 8
The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment
arXiv:2607. 05552v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly issue judgments read as binary verdicts, and a growing literature reports such judgments shifting under logically irrelevant changes of wording - among them an amplified yes-no bias on moral dilemmas, absent in humans.
arXiv Computation and Language
3d ago
Disentangling Question, Response, and Verifier Languages in Multilingual P(True) Self-Evaluation
arXiv:2610.32144v1 Announce Type: new Abstract: When a language model is asked whether its own answer is correct, three languages are involved: that of the question, that requested for the answer, an...
arXiv Computation and Language
Oct 21
Why Do LLMs Accept Generics? Explaining Acceptability using Psycholinguistic and Model-Derived Factors
The paper investigates why large language models (LLMs) accept or reject generic statements, drawing on psycholinguistic research that identifies factors beyond mere property frequency, such as distinctiveness and characteristicness. Using twelve LLMs evaluated on a psycholinguistic dataset and real-world generics, the study finds that models weigh property commonality against distinctiveness similarly to humans, but they prioritize factual truth and largely ignore striking properties. Manipulating hidden states along factor directions confirms that each factor influences judgments as predicted.
Hugging Face Trending Papers
Oct 20
Why Do LLMs Accept Generics? Explaining Acceptability using Psycholinguistic and Model-Derived Factors
Psycholinguistic studies explore why humans accept some generic statements but reject others by showing that these judgments depend on more than how common a property is. People consider several other...
arXiv Machine Learning
6d ago
Prompting Strategy Effects Depend on the Judge: A Multilingual Study of LLM-as-a-Judge Robustness
The study investigates how different prompting strategies affect large language model (LLM) judges across multiple languages and tasks. Using a benchmark of 9 prompting strategies, 4 open-weight judges, 27 languages, and 4 tasks, the authors collected over 1 million pointwise scores validated against human judgments. Results show that prompting effectiveness is highly dependent on the judge model, can reverse direction across judges, and becomes less stable in low‑resource languages, indicating that prompt choice must be tailored to the specific evaluation setting.