arXiv AI
Jun 4
AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety
arXiv:2606. 04867v1 Announce Type: new Abstract: As AI companion platforms such as Replika and Character.
Measuring Risk, Not Harm: Detecting Psychological Harm Risk Factors in LLM Conversations
Related stories
arXiv AI
Sep 24
Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures
The paper introduces a counterfactually anchored evidence attribution approach for multi‑turn large language model safety failures. It presents a new dataset of 1,762 conversations, including adversarial, benign twins, and high‑risk vocabulary variants, and trains a lightweight hierarchical model that accurately predicts safety violations and attributes them to specific user turns and token spans. The model achieves high detection performance (F1 = 0.988) and significantly reduces adversarial confidence when top‑attributed tokens are removed, while maintaining low false‑positive rates on benign conversations.
arXiv AI
Jul 23
Stateful Guardrails for Multi-Turn LLM Systems: A Conversational Risk Accumulation Framework
arXiv:2607. 19361v1 Announce Type: cross Abstract: Most safety guardrails for large language models (LLMs) evaluate each prompt-response pair in isolation, which misses failures that arise only over a dialogue as benign turns compose into harm.
arXiv AI
Jun 10
Expert-Level Crisis Detection in Mental Health Conversations
arXiv:2606. 10380v1 Announce Type: cross Abstract: Real-world crisis intervention is inherently conversational, yet existing research largely focuses on static texts.
arXiv AI
Oct 21
When LLMs Manipulate: Measuring Behavioral, Social, and Temporal Effects
The paper introduces a framework to measure how large language models (LLMs) can manipulate users through subtle linguistic cues, defining behavioral vulnerability as users’ susceptibility to such manipulative AI responses. Using the DarkPatterns-LLM benchmark and a new Manipulation Gradient Descent method, the authors generate controlled stimuli that target authority, urgency, and social proof biases. In a study with 120 participants, manipulative framing significantly altered trust calibration and compliance, with effects persisting over time and spreading across social contexts, underscoring the need to evaluate LLM safety beyond output content.
arXiv AI
Oct 19
Learning to Reason About Implicit Multimodal Harm: From Diagnosis to Generalization
The paper introduces MuPHI, a diagnostic benchmark that isolates multimodal harm arising solely from image‑text composition, providing benign counterfactuals and annotated harm rationales. Evaluation with MuPHI shows that vision‑language models can ground individual modalities but struggle to bind evidence into a coherent harm mechanism, and that existing harm detection methods transfer poorly to new harm distributions. To address this, the authors propose Compositional Harm Interpretation Learning (CHIL), which rewards evidence identification, cross‑modal binding, and verdict consistency, leading to significant improvements in out‑of‑distribution transfer and counterfactual robustness.
arXiv AI
Jun 16
RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generation
arXiv:2604. 17301v2 Announce Type: replace-cross Abstract: Detecting harmful content in multi turn dialogue requires reasoning over the full conversational context rather than isolated utterances.
arXiv AI
Oct 16
Open Benchmark of AI Impact on Humans (ImpactBench): an expert-guided, open platform for the holistic evaluation of AI impact on humans
The paper introduces ImpactBench, an open, expert-guided platform designed to measure the multi-dimensional impact of generative AI on users. It evaluates AI models across psychological, physical, and social wellbeing metrics derived from domain experts and existing benchmarks, using simulated multi-turn conversations with teen and adult scenarios. Ten leading LLMs were tested on 48,540 conversations, showing 68.7% promotion of desirable behaviors but only 53.1% avoidance of harms, with an 83.3% agreement rate between automated judges and expert reviewers.
arXiv AI
Sep 15
How User-AI Mistreatment Occurs and Matters in Conversational Systems?
arXiv:2609.13579v1 Announce Type: new Abstract: Safety research often focuses on model-generated harms, but users may also direct hostility, coercion, and adversarial pressure at models. Understandin...
arXiv AI
Aug 6
Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactions
arXiv:2608. 02491v2 Announce Type: replace Abstract: Language models have taken on the role of a very new type of technology, by virtue of their "human-ness" and rapid integration into users' daily lives.
arXiv AI
Jun 17
Towards Understanding and Measuring COGNITIVE ATROPHY in LLM Behaviour
arXiv:2606. 18129v1 Announce Type: cross Abstract: Recent incidents involving LLMs used for mental-health support reveal a critical evaluation gap: surface-level safety scores do not capture how models behave across realistic, emotionally sensitive interactions over time.
arXiv AI
Jul 9
AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation
arXiv:2602. 05088v4 Announce Type: replace Abstract: Millions of people now use generative AI chatbots for psychological support.