Hugging Face Trending Papers

RELATE: An Evaluation Framework for measuring Relational Orientation of Large Language Models

arXiv AI
Oct 1

FIGS: Evaluating Multi-Turn Sycophancy Without Penalizing Empathy

The paper introduces FIGS, a dual‑axis evaluation framework for multi‑turn sycophancy that avoids penalizing empathy. It uses a 10‑turn conversational simulator with 500 diverse scenarios to test whether models stay truthful while keeping praise proportional, and whether they show calibrated validation of user feelings. The study finds that current models either drift toward sycophancy or become overly detached, highlighting an unresolved trade‑off in sustained dialogue.

By Sidharth Pulipaka, Ruta Binkyte, Ivaxi Sheth, Sahar Abdelnabi
arXiv AI
Sep 11

RESCUE-BENCH: Towards Relation-Aware Multi-Party Emotional Support Conversation Systems

The paper introduces RESCUE-BENCH, a benchmark for relation-aware multi‑party emotional support conversation systems. It is built from real couple and family interview data, comprising 191 samples, 7,079 annotated turns, and 1,064.8 minutes of video, and defines six tasks that assess relational understanding and relation‑sensitive support. Experiments with ten large language models show that while they handle local emotional cues reasonably well, they struggle with tasks that require modeling interpersonal relations, such as predicting relation patterns, viewpoints, and support strategies.

By Haichuan Hu, Yang Xiao, Mingni Tang, Jiawen Duan, Quanjun Zhang, Congqing He, Hao Zhang, Jiashuo Wang, Johan F. Hoorn, Wenjie Li
arXiv AI
Sep 10

A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation

The paper introduces a three-tier persona vector for user simulation in evaluating LLM agents, comprising 23 dimensions across demographics, behavioral traits, and emotional states, plus a query-complexity overlay. It demonstrates that these nuanced personas generate diverse, scenario-reactive conversations, leading to significant variations in agent goal achievement and compliance across different contexts. The model’s design allows for reproducible, auditable user behavior patterns without relying on learned covariance matrices.

By Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu, Mayank Malhotra, Arup Kumar Das, Jitesh Chandra Mishra, Arun Menon, Avinash Karn, Mouli V
arXiv Computation and Language
Sep 15

Sweet Talkers: How Query Formulation Shapes Sycophancy in Romantic Relationship Advice

The paper introduces the Romantic Relationship Advice-Seeking Prompts (RRASP) dataset, comprising 2,400 prompts across five relationship themes, to study how query formulation affects sycophancy in large language models. Using the ELEPHANT framework, the authors evaluated GPT‑5 Mini and Gemini 3 Flash, finding that grammatical mood alone does not drive sycophantic behavior, whereas perspective‑driven framing does, with models increasingly accepting user premises over successive turns. Gemini 3 Flash showed smaller increases in moral sycophancy than GPT‑5 Mini, indicating greater resistance to reinforcing ethically problematic positions.

By Helena Choi, Edric Castel Hao, Karl Bautista, Francis Gabriel Magleo, Renzo Panti, Danielle Beatrice Olalia