P-ContextBench is a benchmark designed to evaluate how well large language models understand and safely use personal context. It creates realistic, life‑story‑grounded digital humans from U.S. demographic seeds, embedding traits, routines, goals, and constraints in fragmented, evolving synthetic mobile‑app records. The benchmark tests five capabilities—fact recovery, temporal tracking, pattern inference, and safe handling of conflicting requests—using a multi‑stage validation pipeline and 4,810 verified test cases.
arXiv:2606.26403v2 Announce Type: replace Abstract: Foundation-model research increasingly needs data about people: user state, personal histories, relationships, contact-like fields, documents, and...
arXiv:2607. 19949v1 Announce Type: new Abstract: Smartphone personal assistants reason over longitudinal personal data, yet evaluating them requires context-rich evaluation data whose correct answers are known, and real device traces are too privacy-sensitive to share.
arXiv:2605. 28969v2 Announce Type: replace-cross Abstract: If an AI agent makes decisions on a person's behalf, those decisions must align with its user.
arXiv:2608. 10042v1 Announce Type: cross Abstract: Tool-use LLMs are increasingly asked to act on users' behalf, but existing benchmarks usually focus on profile recall, style imitation, generic tool use, or response-level personalization.
arXiv:2608. 05246v1 Announce Type: new Abstract: Existing personalized LLM benchmarks primarily rely on textual personas or isolated behavioral signals, providing limited evaluation of cross-domain behavioral personalization, where responses must be grounded in heterogeneous daily-life activities.
ContextEcho is a benchmark and harness designed to measure persona drift in large language models during long, tool‑using coding sessions. It includes a 25‑probe identity suite, a snapshot‑then‑probe protocol that preserves the main conversation, and both judged and judge‑free measurement surfaces. Across 23 frontier models and thousands of turns, the benchmark shows that persona drift is widespread, not limited to specific model families, and that simple in‑session compaction does not reset it, while a single‑shot anchor can restore the intended persona.
arXiv:2608. 16196v1 Announce Type: new Abstract: Personalized game generation requires inferring a player's abilities and behavioral style from how they play.
arXiv:2610.34735v1 Announce Type: new Abstract: An agent that truly knows and acts on behalf of its user has been a long-standing promise of AI assistants. Computer-use agents bring this promise with...
arXiv:2607. 27056v1 Announce Type: new Abstract: Personalized agents are increasingly applied to assist users across a wide range of tasks.
ColaBench is a synthetic benchmark designed to evaluate how personal agents use long‑term interaction history to acquire and apply external knowledge. It contains 349 tasks in health and daily living, 10 user profiles, and a 13,033‑document knowledge base, with an Evidence Graph linking personal conditions, disclosed information, prior document exposure, and answer requirements. The paper also introduces MemKB, a system that uses state summaries and memory‑document links to retrieve relevant knowledge, achieving 86.25% accuracy with an average of 4.07 interaction rounds per task.
The paper introduces the Situated Identity Test (SIT), a framework that assesses whether a language model’s behavior can be traced to a specific developmental lineage rather than merely imitating a persona. SIT requires agents to possess accurate knowledge of their recorded experiences while appropriately ignoring ungrounded information, and it demonstrates that policies based only on compressed profiles are limited in distinguishing between colliding life histories. The authors present SITBench, an evaluation suite with 25 profile‑collision pairs and 10,000 probes across nine model architectures, and provide open‑source tools and pilot results on state‑of‑the‑art foundation models.
The paper introduces a three-tier persona vector for user simulation in evaluating LLM agents, comprising 23 dimensions across demographics, behavioral traits, and emotional states, plus a query-complexity overlay. It demonstrates that these nuanced personas generate diverse, scenario-reactive conversations, leading to significant variations in agent goal achievement and compliance across different contexts. The model’s design allows for reproducible, auditable user behavior patterns without relying on learned covariance matrices.