arXiv Computation and Language September 1, 2026 By Andrew Aquilina, Xiang Lorraine Li, Yu-Ru Li

Whose Assessment of Distress? Community Perspectives and LLM Alignment on Well-Being Posts

Read the original on arXiv Computation and Language →

The study investigates how large language models (LLMs) assess psychological distress in online posts from six identity‑based communities. Through a perspectivist annotation task, 321 participants provided 9,587 judgments on 1,198 Reddit posts, revealing modest in‑group agreement (OR = 1.18) that varies across communities. When evaluated against these community‑specific labels, open‑weight LLMs consistently over‑estimate distress—achieving only 31–44% accuracy on posts perceived as none‑to‑mild—while newer models like GPT‑5 and Gemini 2.5 Pro show similar inflation, whereas Claude Opus 4 is more conservative. "whyItMatters":"The findings highlight that miscalibrated distress detection by LLMs can disproportionately impact the very communities they aim to serve, underscoring the need for equitable AI deployment in mental‑health contexts."

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

Sep 2

LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts

arXiv:2609.00222v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how...

By Daniela Occhipinti, Andrea Piergentili, Marco Guerini

llms safety

arXiv AI

Jun 8

Does Topic Sentiment Cause Perceived Ideology? Comparing Human and LLM Annotations in Political News Articles

arXiv:2606. 06715v1 Announce Type: cross Abstract: We ask whether topic sentiment has a causal effect on perceived political ideology, and whether the answer depends on who assigns the ideology label.

By Upasana Chatterjee

llms fine-tuning

arXiv AI

Jun 12

Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communities

arXiv:2606. 13397v1 Announce Type: cross Abstract: Language operates as a mechanism of both marginalization and resistance, especially for minority communities navigating insensitive and harmful speech online.

By Dipto Das, Achhiya Sultana, Ankit Singh Chauhan, Saadia Binte Alam, Mohammad Shidujaman, Shion Guha, Sunandan Chakraborty, Syed Ishtiaque Ahmed

llms rag

Hugging Face Trending Papers

Aug 31

LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts

Large language models (LLMs) are increasingly used as judges for subjective tasks, where annotators disagree and the relevant question is not only how accurate a judge is, but whose judgments it repro...

llms safety

arXiv Machine Learning

Jun 5

Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability

arXiv:2605. 03217v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "unbiased.

By Yash Aggarwal, Atmika Gorti, Vinija Jain, Aman Chadha, Krishnaprasad Thirunarayan, Manas Gaur

llms efficiency safety

arXiv AI

Sep 2

Some Emotions Run Deeper: Layer-wise Probing and Causal Intervention in Large Language Models

The study examines how emotions are represented across layers of large language models (LLMs) by probing eight 1B–9B open‑weight models on three datasets (Twitter, Reddit, autobiographical narratives). It finds that the optimal probing layer varies systematically with the dataset, moving from near‑input layers to deeper layers, and that targeted forward‑pass interventions on these layers degrade performance more than random interventions. Additionally, the selected layers transfer across datasets and emotion categories, and early‑exit representations from these layers outperform full‑depth exits by an average of 6.9 percentage points.

By Tian Fang, Ga\"el Guibon, Davide Buscaldi

llms

One email a morning, machine-written

Related stories

LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts

Does Topic Sentiment Cause Perceived Ideology? Comparing Human and LLM Annotations in Political News Articles

Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communities

LLM-as-a-Demographic: Whom Sociodemographic Prompting Helps, and Whom It Hurts

Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability

Some Emotions Run Deeper: Layer-wise Probing and Causal Intervention in Large Language Models