Hugging Face Trending Papers
DreamBench-SWE: A Multi-Session Memory-Hygiene Benchmark for Software Agents
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Aug 24
DreamBench-SWE: A Multi-Session Memory-Hygiene Benchmark for Software Agents
DreamBench‑SWE is a multi‑session benchmark designed to evaluate software agents’ memory hygiene, where later tasks rely on evidence from earlier sessions and are scored by executable hidden oracles. The study reports results from an original scaled v2 fold and a preregistered v2.1 successor audit, detailing performance across various memory configurations and conditions. The findings confirm that DreamBench‑SWE can discriminate between different memory setups, but do not establish superiority, equivalence, or broad generality of any external‑system mechanism.
arXiv AI
Aug 25
MemGuard: Persisting Verifier Signals for LLM-Agent Memory Governance
arXiv:2608.21867v1 Announce Type: new Abstract: LLM agents are moving from single-prompt use to long task streams in which reusable memory becomes a core capability for terminal, software-engineering...
arXiv AI
Oct 21
When Does Experience Memory Help Closed-Loop LLM Driving Agents? A Controlled Evaluation with Resampled Warmup Pools, Token-Matched Fillers, and Audited Baselines
arXiv:2610.27714v1 Announce Type: new Abstract: When does experience memory help a closed-loop LLM driving agent? Published gains cannot say, because the two factors we find to matter most, which war...
arXiv AI
Oct 31
When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent Memory
The paper introduces a scale‑conditioned evaluation protocol for agent memory that measures how usable evidence degrades as irrelevant sessions accumulate. It logs memory trajectories and reports four diagnostics—budget‑compliant reliability, tail memory‑call burden, failure‑regime decomposition, and the usable‑scale boundary. Experiments on LongMemEval and LoCoMo show that reliability loss varies across memory interfaces and agents, highlighting that scalable‑memory claims must be conditioned on agent, interface, scale range, and interaction budget.
arXiv AI
Jul 31
MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair
arXiv:2607. 27080v1 Announce Type: cross Abstract: Memory systems allow agents to retain and reuse information from past interactions, but they can also let malicious content persist.
arXiv AI
Sep 16
LSREP: A Longitudinal State-Replay Protocol for Evaluating Conversational Memory, with ICE v2 as an Audited Local-First Architecture
The paper introduces LSREP, a Longitudinal State‑Replay Evaluation Protocol designed to assess how conversational memory evolves over time, incorporating ordered replay, lifecycle schedules, repeated probes, evolving reference answers, and mechanism‑fidelity checks. It applies LSREP to ICE v2, a local‑first memory middleware, and reports that on three ordinary‑density datasets ICE v2 achieves near‑zero mean quality difference from vector‑RAG while using fewer fragments but slightly more prompt tokens, yet fails catastrophically on a dense dataset. In a public diagnostic, ICE v2 underperforms pure vector‑RAG on LongMemEval, revealing significant multi‑session and temporal failures and a quality‑cost trade‑off rather than superior efficiency.