arXiv Computation and Language November 9, 2026 By Haonan Huang

Decomposing Yes-No Bias in LLM Moral Judgment: Stance, Position, Word, and Verdict

Read the original on arXiv Computation and Language →

The paper introduces a comprehensive psychometric battery to dissect the components of yes‑no bias in large language models’ moral judgments. By crossing question polarity, printed order, and answer label, the authors separate stance, position, word, and verdict effects, and fit a two‑parameter model of framing susceptibility and decisiveness. Applying this method to 1.35 million trials across twenty moral dilemmas and ten frontier models, they find that response‑form bias is widespread but varies, with some models exhibiting hidden verdict attachment that surface analysis cannot detect.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Jul 8

The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment

arXiv:2607. 05552v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly issue judgments read as binary verdicts, and a growing literature reports such judgments shifting under logically irrelevant changes of wording - among them an amplified yes-no bias on moral dilemmas, absent in humans.

By Haonan Huang

llms safety

Hugging Face Trending Papers

Jul 6

The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment

Large language models (LLMs) increasingly issue judgments read as binary verdicts, and a growing literature reports such judgments shifting under logically irrelevant changes of wording - among them an amplified yes-no bias on moral dilemmas, absent in humans. A single framing cannot say what such a shift is: in a yes/no question the word "no" is at once logical verdict, lexical token, and last-printed option.

llms safety

arXiv AI

Jul 9

Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas

arXiv:2601. 21433v2 Announce Type: replace Abstract: Language models are increasingly consulted on ethically consequential questions, yet the stance a model expresses may not survive a change in framing.

By Katherine Elkins, Jon Chun

llms safety

arXiv AI

Sep 10

A Layered Analysis of Disagreement And Answer Quality in Multi-Agent LLM Debate

arXiv:2609.08016v1 Announce Type: new Abstract: Multi-agent debate, in which several LLMs exchange arguments before answering, is widely assumed to improve answer quality by surfacing genuine disagre...

By Chen Qian

llms agents safety

arXiv AI

Oct 21

Same Judgment, Different Yes: Measuring Multilingual Judgment Profiles in Language Models

arXiv:2610.28066v1 Announce Type: cross Abstract: Multilingual evaluations often score a model's opening yes or no, a token that also carries model and language answering conventions. We ask when dif...

By Haonan Huang, Yuwei Lu, Chengkun Cai

llms

arXiv Machine Learning

Jun 5

Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability

arXiv:2605. 03217v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "unbiased.

By Yash Aggarwal, Atmika Gorti, Vinija Jain, Aman Chadha, Krishnaprasad Thirunarayan, Manas Gaur

llms efficiency safety

One email a morning, machine-written

Related stories

The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment

The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment

Framing Instability in LLM Ethical Stance: Auditing Negation Sensitivity in Moral Dilemmas

A Layered Analysis of Disagreement And Answer Quality in Multi-Agent LLM Debate

Same Judgment, Different Yes: Measuring Multilingual Judgment Profiles in Language Models

Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability