arXiv AI

The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMs

arXiv:2510. 09905v2 Announce Type: replace Abstract: When an AI assistant remembers that Sarah is a single mother working two jobs, does it interpret her stress differently than if she were a wealthy executive?

arXiv Computation and Language
Sep 18

To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives

The paper introduces ReaLMem, a benchmark built from authentic multi‑year personal visual archives with first‑person annotations, designed to evaluate AI systems on factual recall, persona inference, and predictive personalization. It also proposes ChronoProfiler, a temporal‑weighting module that calculates stability scores for user attributes to resolve preference conflicts and enhance personalized decision making. Experiments with multimodal large language models and memory systems show that predictive personalization remains the hardest task, highlight performance gaps, and demonstrate that temporally informed representations significantly improve personalization.

By Wenqi Zhou, Zhuorui Yu, Kaiao Wen, Hao Zheng, Xinyi Zheng, Peiran Wu, Enmin Zhou, Chi-Hao Wu, Junxiao Shen
arXiv AI
Aug 19

Beyond BFI: The CSI for Enhanced Reliability and Validity in Evaluating LLM Personality Traits

The paper introduces the Core Sentiment Inventory (CSI), a new personality trait evaluation tool for large language models (LLMs) that addresses reliability and validity issues found in existing methods like the Big Five Inventory (BFI). CSI is designed specifically for LLMs, supports both English and Chinese, and provides detailed psychological portraits of model behavior. Experiments show that CSI captures nuanced behavioral patterns, improves reliability, and correlates strongly (above 0.85) with real-world LLM outputs.

By Huanhuan Ma, Haisong Gong, Xiaoyuan Yi, Xing Xie, Philip S. Yu, Dongkuan Xu
arXiv AI
Sep 18

Tailored to you: longitudinal effects of personalising language models

The study examined how personalising language models affects user interactions over five days, comparing a non‑personalised baseline with memory‑based and survey‑based personalisation. Results showed that many interaction changes were due to repeated exposure, but personalisation influenced specific behaviors: memory‑based users disclosed more and found the model less creepy, while survey‑based users felt more regret about sharing personal data. The authors emphasize the nuanced, approach‑specific impacts on user attitudes and the need for careful design of personalised AI.

By Canfer Akbulut, Justine Breuch, Arianna Manzini, Lujain Ibrahim, Matija Franklin, Roma Patel, Iason Gabriel, Kristian Lum, Laura Weidinger
arXiv AI
Sep 2

VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don't Mean Preferences

The paper introduces VIBE‑Bench, a new benchmark designed to test personalized large language models (PLLMs) in a regime where user profile cues and query‑specific preferences do not share the same conceptual space, a situation termed profile‑preference conceptual misalignment (PRCM). VIBE‑Bench contains 3,504 personas, 12,239 dialogues, and a manually verified gold test set, and includes two psychology‑grounded tasks that require cross‑concept preference reasoning beyond surface semantic overlap. Experiments show that existing PLLMs largely depend on shallow semantic correlations and struggle to learn robust cross‑concept mappings, highlighting PRCM as a distinct failure mode for personalization models.

By Yiwen Jiang, Yang Deng, Stephanie Fong, Zimu Wang, Yaling Shen, Wei Feng, Hongxi Yang, Xiangyu Zhao, Zhongxing Xu, Deval Mehta, Xuelian Cheng, Zongyuan Ge
arXiv Computation and Language
Sep 14

PACIFIC: Can LLMs Discern the Psychometric Traits Influencing Your Preferences? Personality-Driven Preference Alignment in LLMs

PACIFIC is a framework that aligns large language model responses with user preferences by leveraging stable Big‑Five personality traits as a latent signal. The authors built a 1,200‑pair dataset covering diverse domains and trait directions, and found that trait‑aligned contexts enable LLMs to achieve near‑ceiling accuracy (up to 99%) in personalized QA. They also introduced a persona‑aware contrastive retriever (PiRAG) that improves label‑free accuracy from 30% to 43% over standard semantic retrieval, highlighting retrieval as the main bottleneck.

By Tianyu Zhao, Siqi Li, Yasser Shoukry, Salma Elmalaki