MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimization
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2606. 28876v2 Announce Type: replace-cross Abstract: We study memory-managed long-context attention: explicit bounded memory with a learned query-independent writer, lifecycle control, query-aware reading, calibrated sparse fallback, and frozen-LLM generation from raw evidence.
The paper introduces LRE (Learned Relevance Eviction), a lightweight, CPU‑only, language‑model‑free scorer that learns which parts of an agent’s interaction history are task‑critical and preserves them verbatim. In experiments, LRE matches or surpasses baseline eviction policies on accuracy‑cost trade‑offs, recovers 93% of full‑history accuracy, reduces worst‑case prompt size by 52%, and outperforms dense and token‑pruning encoders in conversational memory while being 295–1569× smaller. The method also achieves superior budgeted answer quality on LoCoMo reading and can be trained annotation‑free, recovering 95% of supervised scorer performance.
arXiv:2610.15642v1 Announce Type: new Abstract: Under a limited context budget, a long-term agent must recall not merely memories that are individually relevant, but evidence that is collectively suf...
The paper introduces MemCryst, a framework that enables lifelong learning for large language model agents by jointly adapting memory and skills while keeping the language model frozen. MemCryst stores recurring episodes in memory to support skill creation and revision, and uses the resulting skills to guide future interactions, which in turn generate new memory. Experiments across ten diverse settings—including embodied control, long-document reading, conversation, mathematics, and code—show that MemCryst outperforms memory-only and skill-only baselines, achieving 63.17% success on ALFWorld.
Memorilla is a memory system that encodes an entire document collection into reusable embeddings and retrieves query‑specific information as a fixed set of memory tokens for a frozen large language model. Using only 16 memory tokens, it outperforms baselines such as Mem0 and MemoRAG on most benchmarks, achieving higher accuracy on FactKG while drastically reducing token usage. In agentic settings, Memorilla helps prevent wandering and boosts success rates in a TextWorld environment, demonstrating its utility for continual learning and long‑context interactions.
arXiv:2610.33274v1 Announce Type: new Abstract: Large language model (LLM) agents work in scenarios with very different memory needs. A question about past conversations needs the exact facts it name...