arXiv AI September 2, 2026 By Yiwen Jiang, Yang Deng, Stephanie Fong, Zimu Wang, Yaling Shen, Wei Feng, Hongxi Yang, Xiangyu Zhao, Zhongxing Xu, Deval Mehta, Xuelian Cheng, Zongyuan Ge

VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don't Mean Preferences

Read the original on arXiv AI →

The paper introduces VIBE‑Bench, a new benchmark designed to test personalized large language models (PLLMs) in a regime where user profile cues and query‑specific preferences do not share the same conceptual space, a situation termed profile‑preference conceptual misalignment (PRCM). VIBE‑Bench contains 3,504 personas, 12,239 dialogues, and a manually verified gold test set, and includes two psychology‑grounded tasks that require cross‑concept preference reasoning beyond surface semantic overlap. Experiments show that existing PLLMs largely depend on shallow semantic correlations and struggle to learn robust cross‑concept mappings, highlighting PRCM as a distinct failure mode for personalization models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

2d ago

Behaviorally Grounded User Profiles from the Wild for Personalized Alignment and Multi-Perspective Reasoning

arXiv:2609.00014v1 Announce Type: cross Abstract: Persona-driven techniques increasingly adapt large language models (LLMs) to diverse contexts. However, existing methods predominantly rely on rigid,...

By Yuxuan Li, Victor Zhong, Ehsan Kamalloo

llms fine-tuning benchmarks safety

arXiv AI

3d ago

Towards Natural Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions

arXiv:2603.04191v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly serving as personal assistants, where users may share individual preferences over extended interactio...

By Qianyun Guo, Yibo Li, Yue Liu, Bryan Hooi

llms benchmarks

arXiv Machine Learning

Jun 30

Synthetic Interaction Data for Scalable Personalization in Large Language Models

arXiv:2602. 12394v2 Announce Type: replace Abstract: Personalized prompting offers large opportunities for deploying large language models (LLMs) to diverse users, yet existing prompt optimization methods primarily focus on task-level optimization while largely overlooking user-specific preferences and latent constraints of individual users.

By Yuchen Ma, Yue Huang, Wenjie Wang, Xiaonan Luo, Xiangliang Zhang, Stefan Feuerriegel

llms agents reinforcement-learning benchmarks safety

arXiv AI

Jun 8

Re-Centering Humans in LLM Personalization

arXiv:2606. 06614v1 Announce Type: cross Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data.

By Lechen Zhang, Jiarui Liu, Tal August

llms

Hugging Face Trending Papers

Jun 25

SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context

Personalized language-model assistants are often evaluated through a memory lens: can a model recall preferences users have explicitly stated in dialogue? More comprehensive personalization demands a harder capability -- inferring what users care about from the multimodal traces they naturally leave behind.

llms multimodal benchmarks

arXiv Computation and Language

3d ago

Low-Resource Preference Adaptation of LLMs via Activation-Based Label Propagation

arXiv:2608.30902v1 Announce Type: new Abstract: Adapting large language models to user-specific preferences is often constrained by the cost of human annotation, making preference optimisation imprac...

By Alessio Galatolo, Meriem Beloucif

llms safety

One email a morning, machine-written

Related stories

Behaviorally Grounded User Profiles from the Wild for Personalized Alignment and Multi-Perspective Reasoning

Towards Natural Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions

Synthetic Interaction Data for Scalable Personalization in Large Language Models

Re-Centering Humans in LLM Personalization

SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context

Low-Resource Preference Adaptation of LLMs via Activation-Based Label Propagation