arXiv Computation and Language

Controlling and Assessing Appropriate Persona Use in LLM-based Dialogue Generation

The paper examines how large language models (LLMs) tend to overuse persona attributes in persona-based dialogue generation, producing unnatural responses. It identifies a systematic bias in LLMs to incorporate all provided persona details and shows that current metrics cannot assess contextual appropriateness. To address this, the authors introduce Self-CONtrastive Persona Overuse Suppression (SCONPOS), which intervenes in the prompt encoding stage to reduce overuse, and propose the Persona Appropriateness Score (PAS), a new metric that penalizes both overuse and underuse of persona attributes.

arXiv AI
Aug 12

Do LLMs Benefit From Their Own Words?

arXiv:2602. 24287v2 Announce Type: replace-cross Abstract: In multi-turn conversations, large language models typically condition on the full conversation history: both past user prompts and assistant responses.

By Jenny Y. Huang, Leshem Choshen, Wei Sun, Omar Khattab, Ram\'on Fernandez Astudillo, Mehul Damani, Tamara Broderick, Jacob Andreas
arXiv AI
Aug 28

Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference

The paper introduces PRISM, a new framework for evaluating how well large language models (LLMs) maintain persona fidelity in dynamic dialogue. PRISM reframes the task as a structured inverse inference problem grounded in Systemic Functional Linguistics, breaking persona fidelity into Task Framing, Interpersonal Stance, and Linguistic Style dimensions. Experiments demonstrate that PRISM produces more accurate and stable judgments than existing holistic or static psychometric methods, offering a more reliable and auditable evaluation process.

By Mengfan Li, Zesheng Wei, Xuanhua Shi, Yang Deng
arXiv Computation and Language
Sep 23

PERSONAWEAVER: Controllable Diversity Beyond Conventional Archetypes in Procedural Character Generation

PERSONAWEAVER is a new approach to procedural character generation that separates world building from behavioral specification, using manually curated banks of moral positions and conversational reactions to diversify character behavior. By applying this method across ten realistic and fantastical settings and three large language models, the system produces broader moral and interactional response distributions, varied interpersonal language, response length, sentiment, and less archetypal world attribute combinations compared to prior work.

By Maan Qraitem, Kate Saenko, Bryan A. Plummer
arXiv AI
Sep 10

A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation

The paper introduces a three-tier persona vector for user simulation in evaluating LLM agents, comprising 23 dimensions across demographics, behavioral traits, and emotional states, plus a query-complexity overlay. It demonstrates that these nuanced personas generate diverse, scenario-reactive conversations, leading to significant variations in agent goal achievement and compliance across different contexts. The model’s design allows for reproducible, auditable user behavior patterns without relying on learned covariance matrices.

By Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu, Mayank Malhotra, Arup Kumar Das, Jitesh Chandra Mishra, Arun Menon, Avinash Karn, Mouli V
Hugging Face Trending Papers
Sep 8

A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation

The paper introduces a three-tier persona vector to generate diverse, realistic user inputs for evaluating tool-augmented LLM agents. The vector includes 23 dimensions: categorical demographics, continuous behavioral traits, and continuous emotional states, plus a query-complexity overlay. Experiments on 64,698 conversations show that these persona dimensions produce measurable differences in agent performance and realistic scenario-reactive behavior.

arXiv AI
Sep 24

Emergi-PersonaOS: A Persona Agent Operating System for Situational Adaptation and Controllable Evolution

Emergi-PersonaOS is a psychology‑grounded operating system designed to manage persona agents throughout their lifecycle. It structures personas into three layers—dispositional traits, characteristic adaptations, and narrative identity—allowing the system to infer current persona states from situational cues and generate appropriate actions. The OS records experiences, evaluates revision candidates, and controls belief updates through explicit review and traceable evidence, enabling controllable evolution of persona agents over long interactions.

By Haoluan Fu, Keni Chen, Xinyu Jia, Jinpeng Wang, Yuyu Yin