The paper investigates how memory systems can answer a current query correctly yet fail to retain distinctions needed for later updates. Using a paired‑history audit, the authors evaluate 24 history pairs across six synthetic mechanisms and two model backends, achieving perfect reveal accuracy on DeepSeek and high accuracy on GLM. Record‑level audits reveal specific failures in structured reveal memories and frontier late‑reference adequacy, and the authors test a label‑equivariant repair that only partially restores correctness.
arXiv:2608. 08944v1 Announce Type: cross Abstract: A failed retrieval-augmented generation (RAG) answer can be consistent with several unseen responses to evidence repair.
The paper investigates how personalized agents decide to use, ignore, update, or query retrieved user memory before acting on a task. An empirical audit protocol is developed to test structured intermediate outputs, revealing that while exposing state definitions improves accuracy, an explicit state-output field does not significantly enhance policy accuracy for large language models. The study also shows that example-level accuracy overstates consistency, with full four‑way family success being rare, and that providing benchmark‑associated state labels merely conditions predictions rather than proving internal fidelity.
arXiv:2609.22231v1 Announce Type: new Abstract: Long-horizon interactions with LLM-based assistants require memory systems that preserve and update user states, preferences, and interaction histories...
arXiv:2607. 12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions.
The paper introduces LSREP, a Longitudinal State‑Replay Evaluation Protocol designed to assess how conversational memory evolves over time, incorporating ordered replay, lifecycle schedules, repeated probes, evolving reference answers, and mechanism‑fidelity checks. It applies LSREP to ICE v2, a local‑first memory middleware, and reports that on three ordinary‑density datasets ICE v2 achieves near‑zero mean quality difference from vector‑RAG while using fewer fragments but slightly more prompt tokens, yet fails catastrophically on a dense dataset. In a public diagnostic, ICE v2 underperforms pure vector‑RAG on LongMemEval, revealing significant multi‑session and temporal failures and a quality‑cost trade‑off rather than superior efficiency.
The study evaluates an auditable patient‑timeline reconstruction system that tracks provenance, records revisions, and refuses to answer when evidence is missing. Using a synthetic corpus of 1,000 patients and 3,353 notes, two provenance‑aware Evidence Graph operators reduced graph size by 33–37% while preserving all answers across 6,813 query points; a fixed‑window baseline failed to answer over half of the points. The system’s evidence‑gating mechanisms (BioClinicalBERT and a zero‑shot LLM) responded appropriately to evidence‑unavailable controls, but performance varied on marker‑free controls, with BERT maintaining high accuracy but the LLM’s coverage dropping sharply. "whyItMatters":"The results demonstrate that provenance‑aware evidence graphs can significantly reduce data complexity while maintaining answer integrity, highlighting a practical approach to building auditable clinical NLP systems."
The paper identifies a specific issue in supervised fine‑tuning (SFT) of large language models called factual access failure, where models can recognize correct facts under constrained tests but fail to generate them in open‑ended settings. It demonstrates that SFT can cause both genuine wrong answers and expression‑level errors such as verbosity or formatting mismatches. To mitigate this, the authors propose Recall‑Anchored Distillation (RAD), a self‑distillation method that aligns the fine‑tuned model with the base model’s soft output distribution on unlabeled out‑of‑distribution text, thereby recovering lost factual recall without needing labeled data.
arXiv:2608. 19564v1 Announce Type: new Abstract: Persistent memory can personalize an LLM agent, but an incorrect durable update can silently distort future behavior.
arXiv:2607. 26929v1 Announce Type: cross Abstract: The same diagnostic result can support or challenge one causal claim yet fail to address another when the claims concern different populations, outcomes, estimands, pathways, or identifying assumptions.
arXiv:2607. 18867v1 Announce Type: new Abstract: Large language models leak parametric knowledge of realized outcomes into historical financial decision tasks.
The report audits a long‑term‑memory retrieval chain on 500 LongMemEval‑S development questions, evaluating reader performance under an adapted GPT‑4o rubric. Scores vary widely across reader lanes (93–479), with the strongest lanes scoring 479 and 475, and re‑judging the same pass‑1 answers changes three labels, yielding 478. Fixed‑answer knowledge‑update re‑scoring achieves 70/72 or 69/72 depending on the template, and a different‑family reader scores 474, just 1.0 percentage point below the headline pass. The study also reports that live reader request bodies were not retained, that 18 of 23 gains occurred where baseline packets lacked evidence, and that a negative control rejects a verifier that repairs some wrong drafts but breaks many correct ones. Overall, the findings do not establish a new leaderboard leader or a transferable memory advantage, and the released artifacts support packet inspection and re‑scoring but do not reconstruct the method.
arXiv:2608. 10502v1 Announce Type: new Abstract: Persistent memory lets language-model agents reuse information across sessions, but it also makes errors durable: a poisoned, stale, or misattributed record can alter reasoning, tool use, answers, and subsequent memory writes.