The paper introduces ResMem, a method that transforms retrieved knowledge into a parametric memory for large language models. By using the LLM as an offline teacher that reads retrieved text, ResMem learns an additive logit residual, avoiding large token storage and improving question‑answering performance. Experiments on Wikipedia and five QA datasets show significant storage savings and higher token F1 scores compared to k‑NN baselines.
arXiv:2608.17050v3 Announce Type: replace-cross Abstract: Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access...
arXiv:2604. 00715v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) improves language model (LM) performance by providing relevant context at test time for knowledge-intensive situations.
arXiv:2604. 23336v3 Announce Type: replace-cross Abstract: Unlike traditional fact-based retrieval, rationale-based retrieval typically necessitates cross-encoding of query-document pairs using large language models, incurring substantial computational costs.
The paper investigates how Engram-style hashed memory can be transferred between different language model backbones. By freezing a memory table trained on a source model and attaching it to a target model with only a lightweight reader, the authors find that both the memory content and correct addressing are important, but the reader must be aligned to the target to make the memory useful. In question‑answering experiments, a dual‑layer, four‑branch reader nearly matches same‑model performance, and when the reader interface is directly compatible, the frozen memory alone provides substantial benefit, with optional reader adaptation offering further gains.
The paper introduces rEDMRec, a method that compresses a large language model’s reasoning about user preferences and item comparisons into a compact, editable memory. This memory, organized into four channels—long‑term preference, short‑term context, item perception, and counterfactual hard‑negative comparisons—can be updated by an LLM controller and queried by a lightweight student LLM for ranking, eliminating the need to re‑run the expensive teacher model for each request. Experiments on ML‑1M, Amazon Beauty, and Steam datasets show that rEDMRec consistently outperforms zero‑shot, few‑shot, RAG, and GraphRAG baselines, achieving up to a 13.3% improvement in HR@1 on ML‑1M.
Memorilla is a memory system that encodes an entire document collection into reusable embeddings and retrieves query‑specific information as a fixed set of memory tokens for a frozen large language model. Using only 16 memory tokens, it outperforms baselines such as Mem0 and MemoRAG on most benchmarks, achieving higher accuracy on FactKG while drastically reducing token usage. In agentic settings, Memorilla helps prevent wandering and boosts success rates in a TextWorld environment, demonstrating its utility for continual learning and long‑context interactions.
TriageRAG is a failure‑aware self‑training method for retrieval‑augmented language models that adapts them to specialized domains using only unlabeled text. It probes each self‑generated question with and without retrieved context, classifies the result into one of four categories (known, solved, context‑failure, retrieval‑failure), and routes each category to a tailored training signal, including a parametric route that teaches the reader to answer when retrieval misses. Experiments on 11 datasets across three domains and on CS‑Bench show that TriageRAG outperforms a budget‑matched SimRAG by 1.2–6.4 points and uniquely recovers closed‑book performance whenever retrieval harms the base model.
arXiv:2603. 02830v2 Announce Type: replace-cross Abstract: Predicting future student responses to questions is particularly valuable for educational learning platforms where it enables effective interventions.
arXiv:2510.13935v3 Announce Type: replace-cross Abstract: The facts a language model stores are tied to its parameter count, so small models that fit on edge devices fail on expert problems, which ne...
arXiv:2609.36435v1 Announce Type: new Abstract: An assistant that serves the same user over a long horizon has to answer from what that user has revealed: which preferences still hold, which were rev...
arXiv:2609.22100v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves language models with retrieved evidence, but processing many long passages is costly and can introduce...
The paper investigates modular entity disambiguation by separating candidate retrieval from entity selection. It compares sparse retrieval (BM25), Web KB search, and a dense retriever, all paired with large language model selectors. Results show that a training‑free BM25 retriever combined with an LLM selector achieves state‑of‑the‑art performance on the ZELDA benchmark, and the modular approach enables abstention when retrieval fails.