arXiv AI
HABIT-Bench: Evaluating Habit Induction from Longitudinal Weak Evidence in Agent Memory
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Oct 27
BehaviorMem: Building and Evaluating User Memory from Real-World E-Commerce Behavior Streams
arXiv:2610.34790v1 Announce Type: new Abstract: User memory helps personalized systems build on their understanding of users over time. In e-commerce, users convey their preferences and needs through...
arXiv AI
Jul 24
PersonaTrail: Benchmarking Personalized Web Agents through Browsing Trails
arXiv:2607. 20482v1 Announce Type: new Abstract: Recent advances in large language models have enabled web agents to autonomously execute complex tasks.
arXiv Computation and Language
Oct 23
Why Bother Me Again? Benchmarking Personal Agent Memory for Interaction Preferences
The paper introduces MemPABench, an interactive environment designed to evaluate how personal agents remember and act on user interaction preferences. The benchmark features long‑horizon tasks where agents must decide when to clarify, confirm, or act autonomously, using data from four simulated users who provided 99 sessions over four months and five preference changes each. Experiments across four memory systems and LLM backbones show that current memory approaches offer limited improvement in preference selection, with retrieval‑based methods performing no better than no memory and file‑based memory sometimes applying preferences in irrelevant contexts.
arXiv AI
Sep 10
Latent Preference Modeling for Multi-Session Personalized Tool Calling
The paper introduces the Multi-Session Personalized Tool Calling (MPT) benchmark, containing 4,695 instances across 459 multi‑session histories that test Preference Recall, Induction, and Transfer. It proposes PRefine, a test‑time memory method that refines a user’s latent preference via a generate‑verify‑refine loop. Experiments with five LLMs show that PRefine outperforms existing memory systems and even full‑history prompting on Preference Transfer, suggesting that personalized agents should encode behavior as preferences rather than merely storing past interactions.
arXiv AI
Jul 8
From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space
arXiv:2607. 05794v1 Announce Type: new Abstract: Long-term user memory is essential for personalized conversational agents, yet many memory systems still expose memory through passive retrieval interfaces, making the model a consumer of pre-selected evidence.
arXiv Computation and Language
Oct 23
Memory-Aware Reward Modeling: Learning Human Preferences for LLM Memory Use
The paper investigates whether using a language model’s memory to personalize responses aligns with user preferences. By collecting about 1,000 pairwise judgments from 87 participants on ChatGPT queries that included personal memories, the authors find that memory‑conditioned responses are not consistently preferred and that users sometimes strongly favor non‑memory responses. An automatic reward model trained on these judgments achieves 72.8% preference prediction accuracy and outperforms existing baselines, highlighting the need for evaluation and optimization objectives that better reflect user preferences for memory use.