← Back to all news
arXiv AI By Grant Nasir Brown, Abdul Basir Bari, Anmol Tadikonda, Isabel Song

P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior

Read the original on arXiv AI →

P-ContextBench is a benchmark designed to evaluate how well large language models understand and safely use personal context. It creates realistic, life‑story‑grounded digital humans from U.S. demographic seeds, embedding traits, routines, goals, and constraints in fragmented, evolving synthetic mobile‑app records. The benchmark tests five capabilities—fact recovery, temporal tracking, pattern inference, and safe handling of conflicting requests—using a multi‑stage validation pipeline and 4,810 verified test cases.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 12

UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs

arXiv:2608. 10042v1 Announce Type: cross Abstract: Tool-use LLMs are increasingly asked to act on users' behalf, but existing benchmarks usually focus on profile recall, style imitation, generic tool use, or response-level personalization.

By Xuexiong Yin, Zechuan Chen, Yongsen Zheng, Yuxiang Zhang, Jingyuan Yang, Bin Wang, Yubin Wang, Keze Wang
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Aug 7

LUNAR: Benchmarking Personalized Large Language Models on UNiversal User BehAvioR Logs

arXiv:2608. 05246v1 Announce Type: new Abstract: Existing personalized LLM benchmarks primarily rely on textual personas or isolated behavioral signals, providing limited evaluation of cross-domain behavioral personalization, where responses must be grounded in heterogeneous daily-life activities.

By Jiahao Zhang, Yongzhi Tong, Zelin Fu, Pengde Zhao, Yanmei Jiang, Jiang Feng, Min Yang
llmsefficiencybenchmarkssafety
More like this →
arXiv Computation and Language
Aug 28

ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions

ContextEcho is a benchmark and harness designed to measure persona drift in large language models during long, tool‑using coding sessions. It includes a 25‑probe identity suite, a snapshot‑then‑probe protocol that preserves the main conversation, and both judged and judge‑free measurement surfaces. Across 23 frontier models and thousands of turns, the benchmark shows that persona drift is widespread, not limited to specific model families, and that simple in‑session compaction does not reset it, while a single‑shot anchor can restore the intended persona.

By Xianzhong Ding, Yangyang Yu, Changwei Liu, Bill Zhao, Le Chen, Tao Chen
llmsagentsbenchmarks
More like this →