The paper introduces behavioral coherence evaluation, a design‑time method that uses validation evidence from an established instrument to test relationships among outputs of large language models (LLMs). Using the Individual Level Abortion Stigma Scale, the authors prompted five LLMs to complete questionnaires for 627 personas and found that the models scored personas lower on self‑judgment but higher on worries about judgment, often reversing the reference direction for Black personas. Expert review highlighted that disclosure guidance from the models requires context about relationship safety, legal risk, and trusted support.
arXiv:2606. 08483v1 Announce Type: new Abstract: Background: Consumer-facing large language models are now a common source of health information, and they interpret and personalize responses rather than retrieve them.
The paper investigates how large language models (LLMs) can exhibit stigma toward people with psychological conditions by examining their intermediate reasoning steps rather than just final answers. Using clinical expertise, the authors develop a framework to identify and rate stigmatizing language in LLM reasoning, distinguishing between overt prejudice and subtler biases. They also expand an existing mental health stigma benchmark to include more psychological conditions, finding that reasoning analysis reveals far more stigma than traditional multiple-choice evaluations and exposes flaws in the models’ logic and understanding of mental health.
arXiv:2608. 05583v1 Announce Type: cross Abstract: As large language models (LLMs) enter high-stakes domains such as healthcare, understanding their moral reasoning becomes essential.
As large language models (LLMs) enter high-stakes domains such as healthcare, understanding their moral reasoning becomes essential. Decisions about scarce medical resources often hinge on judgments of responsibility, particularly when patients' own actions contribute to illness.
arXiv:2609.38036v1 Announce Type: cross Abstract: Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support tools with...
arXiv:2609.38036v2 Announce Type: replace-cross Abstract: Understanding gender biases in large language models (LLMs) is increasingly important as these systems become embedded in decision-support to...
arXiv:2609.15849v1 Announce Type: cross Abstract: Can LLMs reason through new information like humans, or do they merely retrieve cached opinions? This is critical for silicon sampling, where LLM per...
How can we evaluate whether frontier AI systems recognize child-safety risks before they escalate into explicit harm? Existing child safety evaluations focus on child sexual abuse material, yet many child-safety failures begin earlier: in model assistance that helps adults manipulate, impersonate, profile, or isolate minors, and in model responses that deepen children's emotional dependence on AI systems rather than redirecting them toward human support.
arXiv:2608. 15254v1 Announce Type: new Abstract: Clinical-AI guidance increasingly recommends prompting language models to reason with attention to diversity, equity, and inclusion (DEI).
arXiv:2609.01548v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly used in advice seeking and decision making that may affect social judgements. Despite stigma's profound e...
The paper introduces a novel framework for assessing second‑order bias in large language models (LLMs), defined as bias in how an LLM judges the acceptability of biased content. Using principles from entitlement epistemology, the authors design a reasoning task that asks LLMs to determine whether a biased text is acceptable for specific demographic groups, and propose two metrics to quantify biased judgments. Experiments on both open‑source and closed‑source models reveal that the task bypasses safety guardrails, uncovers systematic variations across target groups, and demonstrates that models still rely on demographic labels when evaluating bias.
Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how accurate a judge is, but whose judgments it repro...