arXiv:2606. 30256v1 Announce Type: new Abstract: Safety benchmarks often buy scalability by fixing the prompt, the language, and the turn structure.
By Camilo Chac\'on Sartori
The paper analyzes how large language models (LLMs) maintain consistency when responding to their own generated text in emotionally charged conversations. By treating LLM-generated responses as queries and evaluating the subsequent replies, the study examines three false claim queries across extreme and moderate emotional intensities. Results show that commercial models (Claude‑3.5‑haiku, GPT4o‑mini) and a medium‑sized model (Mistral‑7B) perform below average and are especially vulnerable to false beliefs in moderate emotional contexts, with an attention‑score analysis revealing a shift from evaluative to generative priorities.
By Sneha Oram, Ojaswita Bhushan, Tejomay Kishor Padole, Suyash P. Awate, Pushpak Bhattacharyya
arXiv:2605. 22714v3 Announce Type: replace Abstract: Large language models are routinely used as automated evaluators: to review code, moderate content, or score outputs, often with many items passing through one conversation.
By Sid-Ali Temkit
arXiv:2608.21242v1 Announce Type: new
Abstract: As conversational companions, large language models (LLMs) often have access to users' emotional states. We study how this affective context modulates...
By Jiayi Li, Sanjana Menon, Brett Frischmann, Shomir Wilson, Sarah Rajtmajer
arXiv:2605. 03217v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "unbiased.
By Yash Aggarwal, Atmika Gorti, Vinija Jain, Aman Chadha, Krishnaprasad Thirunarayan, Manas Gaur
The paper introduces DistressBench, a benchmark comprising 718 English clinician-authored conversations covering 23 self-harm subcategories, each evaluated against a weighted rubric across seven crisis-support dimensions. Models are scored by the fraction of criteria they satisfy, revealing that the best models meet 88.4% of weighted criteria while clinicians achieve 98.7%. The study highlights gaps in crisis response, notably that 35.3% of conversations where crisis is recognized fail to route users to human help, underscoring the need to evaluate models on supportive care rather than mere refusal.
By Drew Rein, Patrick Oathout, Vishal Kumar, Udari Madhushani Sehwag