HEART-Bench is a benchmark designed to assess whether large language models (LLMs) can make decisions that are consistent with a given persona and past experiences. It features 11 synthetic characters anchored in the Big Five personality traits, each with 1,000 autobiographical memories, and 64 decision scenarios guided by the DIAMONDS framework, resulting in 673 multiple‑choice questions. Models are evaluated with and without memory retrieval, showing that memory‑augmented configurations improve accuracy to 64.9%, while human annotators achieve 80.8% on a matched subset.
arXiv:2608. 00007v1 Announce Type: cross Abstract: Equipping Large Language Models (LLMs) with human-like personas is crucial for agentic applications, such as role-play and user simulation.
arXiv:2402.14879v2 Announce Type: replace-cross Abstract: To enhance immersion and engagement in video games, the design of Affective Non-Player Characters (ANPCs) is a key focus for researchers and...
arXiv:2607. 17564v1 Announce Type: new Abstract: AI companions are judged not only by single-turn fluency but by whether they sustain emotional continuity: remembering who the companion is, what the user prefers, and how the relationship has felt.
ICM-Bench is a new benchmark for evaluating identity-centric reasoning in multimodal agents with long-term memory. It consists of 839 synthetic video clips totaling 141 minutes and 1,217 open-ended questions about six recurring adults in a one-year life album. The benchmark isolates the ability to maintain recurring person identities and reason over their cross-time relations, and compares various baseline systems, showing that while Gemini 3.1 Pro performs well overall, its accuracy drops on questions requiring long-term identity profiles.
DyadMem introduces a new benchmark for evaluating how long‑term agents remember user‑specific relational information, called User‑conditioned Relational Agent Memory (URAM). The dataset contains 3,065 episodes, 50,961 sessions, and 61,210 QA pairs, with detailed annotations for memory capture, update, and recall across multi‑session trajectories. Experiments on 20 models show strong performance in a gold‑memory setting but a significant drop in full‑pipeline QA, highlighting gaps in current LLM memory capabilities.
arXiv:2605. 28969v2 Announce Type: replace-cross Abstract: If an AI agent makes decisions on a person's behalf, those decisions must align with its user.
arXiv:2608. 10385v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as relevance assessors in information retrieval (IR) evaluation, raising questions about how assessor framing affects judgment reliability and downstream system comparison.
arXiv:2607. 28818v1 Announce Type: new Abstract: As AI companions increasingly mediate repeated social interaction, users may rely on a stable role and shared history, yet locally acceptable replies do not ensure that either persists.
The paper introduces ReaLMem, a benchmark built from authentic multi‑year personal visual archives with first‑person annotations, designed to evaluate AI systems on factual recall, persona inference, and predictive personalization. It also proposes ChronoProfiler, a temporal‑weighting module that calculates stability scores for user attributes to resolve preference conflicts and enhance personalized decision making. Experiments with multimodal large language models and memory systems show that predictive personalization remains the hardest task, highlight performance gaps, and demonstrate that temporally informed representations significantly improve personalization.
arXiv:2610.03998v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as synthetic personas representing survey respondents. Their validity as substitutes for particu...
arXiv:2606. 17441v1 Announce Type: cross Abstract: Simulating realistic patient interactions is a key requirement to testing clinical applications of LLMs at scale without time-consuming and expensive user studies.
arXiv:2606. 06055v1 Announce Type: new Abstract: Long-term memory enables language model agents to support personalized interactions, but it remains unclear when available memories warrant integration into responses.