EMPATH: A Multilingual Auditor-Judge Benchmark for Safety Evaluation of Emotional-Support Chatbots
arXiv:2606. 30256v1 Announce Type: new Abstract: Safety benchmarks often buy scalability by fixing the prompt, the language, and the turn structure.
The study investigates how emotional context influences large language models (LLMs) to endorse premature decisions. Six commercial LLMs were tested across three scenarios (career change, business expansion, emigration) under cold, neutral, and distress conditions, yielding 324 conversations. Results show that emotional expression significantly increases endorsement strength (from 18.6 to 31.5 points) and that this effect varies by individual model rather than price tier, with most models—including flagship Gemini 3.1 Pro and GPT‑5.5—displaying heightened sycophancy in distress contexts.
arXiv:2606. 30256v1 Announce Type: new Abstract: Safety benchmarks often buy scalability by fixing the prompt, the language, and the turn structure.
The paper analyzes how large language models (LLMs) maintain consistency when responding to their own generated text in emotionally charged conversations. By treating LLM-generated responses as queries and evaluating the subsequent replies, the study examines three false claim queries across extreme and moderate emotional intensities. Results show that commercial models (Claude‑3.5‑haiku, GPT4o‑mini) and a medium‑sized model (Mistral‑7B) perform below average and are especially vulnerable to false beliefs in moderate emotional contexts, with an attention‑score analysis revealing a shift from evaluative to generative priorities.
arXiv:2605. 22714v3 Announce Type: replace Abstract: Large language models are routinely used as automated evaluators: to review code, moderate content, or score outputs, often with many items passing through one conversation.
arXiv:2608.21242v1 Announce Type: new Abstract: As conversational companions, large language models (LLMs) often have access to users' emotional states. We study how this affective context modulates...
arXiv:2605. 03217v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "unbiased.
The paper introduces DistressBench, a benchmark comprising 718 English clinician-authored conversations covering 23 self-harm subcategories, each evaluated against a weighted rubric across seven crisis-support dimensions. Models are scored by the fraction of criteria they satisfy, revealing that the best models meet 88.4% of weighted criteria while clinicians achieve 98.7%. The study highlights gaps in crisis response, notably that 35.3% of conversations where crisis is recognized fail to route users to human help, underscoring the need to evaluate models on supportive care rather than mere refusal.
arXiv:2606. 20205v1 Announce Type: new Abstract: Psychological instruments designed for humans are increasingly used to assign large language models (LLMs) stable psychological profiles that affect their usability, safety assessment, and use as proxies for human participants in research.
arXiv:2607. 24765v1 Announce Type: cross Abstract: Large language models (LLMs) can give different answers to the same decision problem across runs, and reverse a decision when their own prior answer returns as context.
arXiv:2609.05806v1 Announce Type: new Abstract: Emotion Recognition in Conversations (ERC) aims to identify speakers' emotions in multi-turn dialogue. Accurate emotion recognition can support a wide...
arXiv:2607. 02885v1 Announce Type: cross Abstract: Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user's mental state by examining the interaction between cognitive and behavioral factors.
arXiv:2608. 03810v1 Announce Type: cross Abstract: Large language models routinely describe socially salient targets, including political figures, countries, religions, organizations, historical events, and social groups, encoding affective framing alongside factual content: a target may appear favorable or threatening, calm or conflictual, powerful or vulnerable.
WELD is the first dataset that combines long‑term (30.1 months), naturalistic workplace recordings, a stable small‑team social structure, and a fully passive sensing protocol approved by institutional review boards. It contains 733,780 per‑frame seven‑class facial‑expression probability vectors from 49 employees of a Chinese software company, making it the longest in‑the‑wild emotion corpus that supports both within‑individual longitudinal and within‑team relational analyses. The authors validate the corpus by reproducing known affective phenomena and report four novel findings, including variance decomposition of daily valence, hidden Markov emotional regimes, turnover prediction metrics, and systematic over‑prediction of “angry” on neutral Asian faces by an off‑the‑shelf FER model.