arXiv AI By Deepesh Sonar

LSREP: A Longitudinal State-Replay Protocol for Evaluating Conversational Memory, with ICE v2 as an Audited Local-First Architecture

Read the original on arXiv AI →

The paper introduces LSREP, a Longitudinal State‑Replay Evaluation Protocol designed to assess how conversational memory evolves over time, incorporating ordered replay, lifecycle schedules, repeated probes, evolving reference answers, and mechanism‑fidelity checks. It applies LSREP to ICE v2, a local‑first memory middleware, and reports that on three ordinary‑density datasets ICE v2 achieves near‑zero mean quality difference from vector‑RAG while using fewer fragments but slightly more prompt tokens, yet fails catastrophically on a dense dataset. In a public diagnostic, ICE v2 underperforms pure vector‑RAG on LongMemEval, revealing significant multi‑session and temporal failures and a quality‑cost trade‑off rather than superior efficiency.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
3d ago

When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent Memory

The paper introduces a scale‑conditioned evaluation protocol for agent memory that measures how usable evidence degrades as irrelevant sessions accumulate. It logs memory trajectories and reports four diagnostics—budget‑compliant reliability, tail memory‑call burden, failure‑regime decomposition, and the usable‑scale boundary. Experiments on LongMemEval and LoCoMo show that reliability loss varies across memory interfaces and agents, highlighting that scalable‑memory claims must be conditioned on agent, interface, scale range, and interaction budget.

By Jiaqi Shao, Yiyi Lu, Yunzhen Zhang, Bing Luo
arXiv AI
Aug 19

D$^2$ACCI: A Dual-Loop Diagnostic Protocol for Evidence-Preserving Agent Memory

The paper introduces D$^2$ACCI, a dual-loop diagnostic protocol designed to improve evidence-preserving memory in large language model agents. It provides a structured framework that uses paired evidence, protected-slice monitoring, and trace-level localizability to decide whether to promote, flag, or reject memory interventions. The authors also present DCR, a metric for measuring failure localizability, and D$^2$ACCI‑Eval, a reusable artifact for gate replay, demonstrating significant performance gains on three public benchmarks and highlighting the importance of traceable, statistically grounded diagnostics.

By Xule Liu, Yijun Liu, Chao Li, Shao Kun
Hugging Face Trending Papers
Aug 18

D$^2$ACCI: A Dual-Loop Diagnostic Protocol for Evidence-Preserving Agent Memory

The paper introduces D$^2$ACCI, a dual-loop diagnostic protocol designed to improve evidence-preserving memory in large language model agents. It provides stage-level diagnostic traces, protected-slice monitoring, and a graded observability metric (DCR) to localize failures within the memory pipeline. Applied to MemStack and evaluated on three benchmarks, the protocol achieves high accuracy and demonstrates statistically significant gains from specific memory interventions.