Evaluating Personal Information Output from Conversational Interactions in Generative AI Systems
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2608.28833v1 Announce Type: new Abstract: While Large language models (LLMs) incorporate user personalization signals to improve usability and helpfulness, they increasingly shift from providin...
The study examined how personalising language models affects user interactions over five days, comparing a non‑personalised baseline with memory‑based and survey‑based personalisation. Results showed that many interaction changes were due to repeated exposure, but personalisation influenced specific behaviors: memory‑based users disclosed more and found the model less creepy, while survey‑based users felt more regret about sharing personal data. The authors emphasize the nuanced, approach‑specific impacts on user attitudes and the need for careful design of personalised AI.
The paper introduces a three-tier persona vector for user simulation in evaluating LLM agents, comprising 23 dimensions across demographics, behavioral traits, and emotional states, plus a query-complexity overlay. It demonstrates that these nuanced personas generate diverse, scenario-reactive conversations, leading to significant variations in agent goal achievement and compliance across different contexts. The model’s design allows for reproducible, auditable user behavior patterns without relying on learned covariance matrices.
arXiv:2601. 11049v2 Announce Type: replace-cross Abstract: We examine whether large language models (LLMs) can predict biased decision-making in conversational settings, and whether their predictions capture not only human cognitive biases but also how those effects change under cognitive load.
arXiv:2607. 27056v1 Announce Type: new Abstract: Personalized agents are increasingly applied to assist users across a wide range of tasks.
The paper investigates Self‑Generated Text Recognition (SGTR), the ability of large language models (LLMs) to identify their own outputs. By evaluating 13–21 models across 6 experimental designs, it shows that SGTR accuracy varies with evaluation format, conversation structure, and task domain, and that a quality‑heuristic bias dominates results. The study also finds that fine‑tuning for SGTR in one setting can generalize to others and may cause models to prefer their own outputs when judging, highlighting potential safety concerns.