The paper investigates how conversational Retrieval-Augmented Generation (RAG) benchmarks that rely on relevance judgments to determine context sufficiency can be misleading. By re‑running official baselines on the RETECO track and evaluating with both large‑language‑model judges and a blind audit, the authors show that stronger retrievers produce a higher proportion of contexts lacking gold documents yet still containing the answer, leading to a sharp drop in shortcut precision from 0.68 to 0.10. The study demonstrates that many false misses arise from hard negatives, gold content from other turns, or neighboring chunks, and that using retrieval scores alone as a sufficiency detector can underestimate answerability by up to 0.19 in AUROC.
By Ivan Kochergin, Vasily Konovalov
arXiv:2609.25054v1 Announce Type: new
Abstract: For a long-horizon LLM agent, the memory question is not what was once recorded but what \emph{currently holds}. Most designs answer it only indirectly...
By Bowen Qin, Yao Lu
arXiv:2606. 01435v1 Announce Type: new Abstract: LLM-based memory systems increasingly maintain facts that evolve over time, where a recurring failure is conflict resolution: when a fact has multiple contradictory values, which should the agent return?
By Vikas Reddy, Sumanth Challaram
arXiv:2608. 08055v1 Announce Type: new Abstract: Large language model (LLM) agents that assist users over weeks of conversation must remember what is currently true, not merely what was once said.
By Fengrong Wan, Chengcan Wu, Ningtao Lyu
A transformer language model performs a bounded amount of computation per token, and recent work by Vishal Sikka, former CEO of Infosys, argues that this bound limits which tasks a model can carry out...
The paper introduces Galahad, a memory layer that stores a transformer language model’s key‑value state for blocks of text, allowing subsequent requests to reuse previously computed attention rather than recomputing it. On seven real‑world datasets, 98.7% of prompt tokens were already read, and with Galahad the model could attend to an entire 97,000‑token corpus, achieving 98–100% recall on a 100‑fact test while reducing inference time and energy consumption dramatically. The approach was validated across 30 models and all runtimes, demonstrating that stateful inference can replace stateless serving without loss of accuracy.
By Sietse Schelpe
arXiv:2606. 09900v1 Announce Type: cross Abstract: Long-term memory is the missing layer for LLM agents: across sessions they forget, and the common workaround -- replaying the whole history into the prompt -- is expensive, slow, and, as distractors accumulate, less accurate.
By Liuyin Wang
Every memory-based knowledge editor in the SERAC lineage depends on a scope decision: given a query, does a stored edit apply? We report that current knowledge-editing benchmarks cannot measure this d...
arXiv:2609.08279v1 Announce Type: cross
Abstract: Agent memory systems must discard stored information when their history exceeds a fixed token budget. Existing budget-accuracy frontiers quantify the...
By Chen Shen
Mnemon is a memory agent that stores conversations as raw, dated records and uses a fast System 1 decision model (Jev) to quickly judge the relevance of records, while a slow System 2 LLM plans searches and composes answers. The agent consolidates records into topic timelines and value histories in the background, enabling efficient retrieval without rewriting conversations into structured formats. Experiments show Mnemon achieving high scores on LoCoMo and LongMemEval‑S with low context length and cost, and Jev outperforming LLMs in evidence separation and speed.
By Guangren Wang
arXiv:2609.18154v1 Announce Type: cross
Abstract: We describe our system for LitTraceQA (GroundLM @ EMNLP 2026): given a research question, retrieve the relevant papers from a pool of 27,487, cite th...
By Aaditya Chauhan
The study shows that in memory‑agent benchmarks, simply retrieving the correct fact does not guarantee accurate answers. On the FactConsolidation task, increasing the evidence budget from 64 to 512 tokens caused accuracy to drop, even though the newest statement was present in every retrieved set. Experiments with Qwen models revealed that replacing an unrelated statement with an obsolete competing statement drastically reduced accuracy, indicating that outdated information can be more harmful than irrelevant evidence.
By Oleeviya Babu Poikarayil, Tony Basil Babu, Abdulrahman Nahhas, Het Darshan Mehta, Jawid Ahmad Baktash, Ernesto William De Luca, Klaus Turowski