arXiv AI
Jun 30
Cognitive World Models for Process-Level Social Influence Evaluation
arXiv:2606. 29495v1 Announce Type: new Abstract: Social influence dialogue changes user behavior by altering internal cognitive states.
Cognitive World Models for Process-Level Social Influence Evaluation
Social influence dialogue changes user behavior by altering internal cognitive states. The central evaluation question is whether the user's beliefs, desires, intentions, and emotions measurably change over the course of conversation, a process-oriented criterion that neither surface-level text metrics (BLEU/ROUGE) nor single-score LLM judgments can capture.
Related stories
arXiv AI
Jun 18
Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts
arXiv:2606. 18258v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit a wide range of human-like behaviors, from expressing thoughts and emotions, to engaging in relationship-building with users, to refusing requests and maintaining boundaries.
arXiv AI
Jun 15
OdysSim: Building Foundation Models for Human Behavior Simulation
arXiv:2606. 14199v1 Announce Type: cross Abstract: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation.
arXiv AI
Aug 3
Cognitive World Model for Progressive BDI/E Trajectory Evaluation of Conversational Agents
arXiv:2606. 29495v2 Announce Type: replace Abstract: As LLM-based conversational agents advance toward increasingly open-ended and interaction-intensive scenarios, task completion alone provides an incomplete assessment of their effectiveness.
Hugging Face Trending Papers
Sep 16
When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AI
Emotion recognition in conversation (ERC) is a production capability behind agent-assist prompts, escalation routing, and post-call analytics in contact-center-as-a-service (CCaaS) platforms, where co...
arXiv AI
Jun 3
Decomposing and Measuring Evaluation Awareness
arXiv:2605. 23055v2 Announce Type: replace-cross Abstract: Frontier language models sometimes recognize that they are being evaluated and adjust their behavior, undermining validity of benchmark results.
arXiv AI
Sep 17
When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AI
The paper evaluates three approaches for emotion recognition in conversation— a low‑cost stacked ensemble, an off‑the‑shelf LLM prompt, and a confidence‑gated hybrid that escalates only uncertain ensemble predictions to the LLM. Across three datasets (IEMOCAP, MELD, CMU‑MOSI), the hybrid consistently outperforms each pure system, achieving higher weighted F1 scores while routing most traffic through the inexpensive ensemble. This results in significant cost savings (≈$10‑85 per million utterances) and provides an interpretable escalation signal tied to emotion or sentiment shifts.
arXiv AI
Aug 11
Emotion2Skill: Model-Internal Emotion Signals for Adaptive Skill Selection and Evolution
arXiv:2608. 09248v1 Announce Type: new Abstract: Skill-based LLM agents select reusable procedures from an external library to solve complex tasks, yet their routing decisions rely entirely on text-level signals such as task descriptions, verbal reflections, and experience-derived rules, while the model's own internal representational state remains unobserved.
arXiv Computation and Language
Aug 24
ZenGen: Social Mind for LLMs
ZenGen is a comprehensive framework designed to develop socially intelligent large language models (LLMs). It introduces SoMBench, a psychology‑grounded benchmark with 3 primary and 17 secondary dimensions, and shows that current LLMs perform poorly on social cognition tasks. ZenGen combines supervised fine‑tuning, on‑policy distillation, and rubric‑based reinforcement learning to improve internalization, while Actio provides deployment‑time grounding with typed supports such as PRISM, Starling, SAGE, and gated RAG, yielding significant performance gains across benchmarks.
arXiv AI
4d ago
PADM\'E: Preference Alignment Data Synthesis for Meta-Evaluation of LM Agent Evaluators
PADM'E is a method for synthesizing preference‑aligned data to meta‑evaluate language‑model (LM) evaluators of agentic behaviors. It reframes meta‑evaluation as a preference judgment problem, generating criterion‑based data with small LMs and no human involvement. In a prototype, PADM'E produced 1,000 samples across four domains and three criteria, and human validation showed agreement with human judgment rising from 73% to 85% compared to a naive baseline.
arXiv AI
Jun 10
AMEL: Accumulated Message Effects on LLM Judgments
arXiv:2605. 22714v3 Announce Type: replace Abstract: Large language models are routinely used as automated evaluators: to review code, moderate content, or score outputs, often with many items passing through one conversation.
arXiv AI
Jul 7
The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Models
arXiv:2604. 19139v3 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) continue to evolve through alignment techniques such as Reinforcement Learning from Human Feedback (RLHF) and Constitutional AI, a growing and increasingly conspicuous phenomenon has emerged: the proliferation of verbal tics--repetitive, formulaic linguistic patterns that pervade model outputs.