arXiv AI By Jiajun Chen, Xinyue Chen, Yicheng Jiang, Hua Shen

Knowing the User Is Not Knowing What We Established: Grounding Awareness in Personal Agents

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv Computation and Language
Sep 21

MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks

MemoryArena is a new evaluation gym that benchmarks agent memory in interdependent multi‑session tasks. Unlike prior benchmarks that test memorization or single‑session action in isolation, MemoryArena requires agents to acquire memory while interacting with the environment and then use that memory to guide future decisions across a range of tasks such as web navigation, planning, information search, and formal reasoning. The benchmark reveals that agents excelling on existing long‑context memory tests perform poorly here, highlighting a gap in current memory evaluation methods.

By Zexue He, Yu Wang, Churan Zhi, Yuanzhe Hu, Tzu-Ping Chen, Lang Yin, Ze Chen, Tong Arthur Wu, Siru Ouyang, Zihan Wang, Jiaxin Pei, Julian McAuley, Yejin Choi, Alex Pentland
arXiv AI
Sep 3

AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents

AdaMem introduces adaptive memory policies that allow personalized agents to decide what information to write into long‑term memory based on user preferences for each interaction context. Each policy is updated from periodic feedback and controls subsequent memory writing, aiming to improve relevance and reduce unnecessary memory persistence. In experiments on AdaMem‑Bench, AdaMem raises QA accuracy from 80.0% to 84.35% while cutting persistent memory by 9.27%, though models still struggle to execute policies reliably.

By Xingyu Chen, Rui Wang, Zhaopeng Tu, Liefeng Bo
arXiv AI
Oct 19

Testing the Planning Claims of LLM-Based Agents against Formal Semantics

The paper evaluates the planning abilities of large language model (LLM) agents by comparing their performance against formal semantics defined in hidden PDDL models. Across five domains, four LLMs, six prompting strategies, and three agent architectures, the best setup solved 78% of solvable tasks offline but only 40% online, with limited recognition of unsolvable tasks and no clear benefit from more elaborate prompting. The study concludes that claims of LLM planning competence require validated action traces rather than end‑to‑end success rates.

By Oriol Miro-Lopez-Feliu, Oscar Mir\'o L\'opez-Feliu, Victor Gimenez-Abalos, Javier Vazquez-Salceda, Victor Carballo Araruna, Sergio Alvarez-Napagao