arXiv Computation and Language By Priyanka Dey, Rajarshi Das, Vikas Bahirwani, Qi Zeng, Julia Kruk, Qi Qi

Adaptive Personalization Under Preference Epistemic Uncertainty

Read the original on arXiv Computation and Language →

The paper introduces the concept of adaptive personalization under preference epistemic uncertainty, arguing that conversational AI must adjust its response strategy as human preferences evolve. A study shows people prefer state‑contingent strategies when uncertainty changes, and the authors build a multilingual benchmark to test language models on this task. Results reveal a recognition‑generation gap: models can choose the right strategy from options but fail to generate it correctly, defaulting to exploit‑first behavior; targeted training on internal strategy representations improves performance on out‑of‑distribution personas.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language
Aug 27

Learning What to Share and What to Personalize: Hierarchical Strategy Co-Evolution for Agent Memory

The paper introduces HiPS, a hierarchical strategy co‑evolution framework for memory‑augmented agents that separates memory management into a globally shared foundation and a user‑specific adaptive tier. HiPS uses a Universal Strategy to capture shared principles from cross‑persona trajectories, Persona Delta Distillation to create tailored rules for users deviating from general patterns, and Cross‑Level Rule Flow to dynamically adjust the boundary between global and personal rules. Experiments show that this approach consistently outperforms existing memory‑augmented baselines.

By Yupeng Han, Shuochen Liu, Kai Zhang, Ze Liu, Zhihong Pan, Xianquan Wang
arXiv Computation and Language
Oct 23

Memory-Aware Reward Modeling: Learning Human Preferences for LLM Memory Use

The paper investigates whether using a language model’s memory to personalize responses aligns with user preferences. By collecting about 1,000 pairwise judgments from 87 participants on ChatGPT queries that included personal memories, the authors find that memory‑conditioned responses are not consistently preferred and that users sometimes strongly favor non‑memory responses. An automatic reward model trained on these judgments achieves 72.8% preference prediction accuracy and outperforms existing baselines, highlighting the need for evaluation and optimization objectives that better reflect user preferences for memory use.

By Yijiang River Dong, Zexue He, Cheng Yu, Tiancheng Hu, Michael J Ryan, Jan Batzner, Zheng Hui, Shaolong Wu, Orestis Papakyriakopoulos, Alex Pentland, Nigel Collier, Jiaxin Pei