KVMem is a KV-context virtualization system that allows large language model agents to maintain workspaces exceeding both GPU key‑value capacity and the model’s native context window. It stores overflowed history as paged KV state across GPU memory, host memory, and NVMe, using lightweight, model‑native attention‑space indexes to retrieve relevant historical blocks. Evaluations on long‑context agent benchmarks show that KVMem improves task utility and inference efficiency, enabling up to one million‑token workspaces on consumer GPUs and achieving interactive responsiveness in local deployments.
arXiv:2607. 27090v1 Announce Type: cross Abstract: Large language models are increasingly deployed with persistent personalized context, such as accumulated memory profiles or long conversation histories, that is shared across a user's many requests.
Large language models are increasingly deployed with persistent personalized context, such as accumulated memory profiles or long conversation histories, that is shared across a user's many requests. Production memory systems (e.
arXiv:2608.21690v1 Announce Type: new Abstract: LLM agents increasingly take on long-running tasks whose history grows far beyond a single model context window. Existing approaches compress earlier i...
arXiv:2606. 06256v1 Announce Type: new Abstract: As the input length of large language model (LLM) serving continues to grow, the KV cache has become a dominant bottleneck in AI infrastructure.
arXiv:2609.34060v2 Announce Type: replace Abstract: Prompt-specialized multi-agent systems enable multiple agents to share a model while performing complementary roles to solve complex tasks. However...
arXiv:2606. 02964v1 Announce Type: cross Abstract: Large Language Model (LLM) inference relies on key-value (KV) caches to avoid redundant attention computation.
LLM agents increasingly take on long-running tasks whose history grows far beyond a single model context window. Existing approaches compress earlier interactions or extract selected information into...
arXiv:2608. 15127v1 Announce Type: cross Abstract: Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and persistent state.
arXiv:2610.19822v1 Announce Type: cross Abstract: LLM agents are evolving from a single model calling tools to planners that orchestrate multiple expert models whose capabilities live in distinct wei...
The paper presents a memory layer called galahad-kv that stores the key-value state of a large language model’s context blocks (≈16,000 tokens each) on encrypted local NVMe disk, allowing byte‑exact retrieval without recomputation. On a single NVIDIA H100, the authors tested the system with Gemma 4 12B and 31B models over 50 million tokens of real public text, achieving 100 % successful reloads, 2.8×–4.3× faster loading than recomputation, and 8.8×–12.3× lower GPU energy usage while keeping GPU memory flat. The models answered long‑range factual queries with high accuracy (82 % for 12B, 98 % for 31B) and did not hallucinate, demonstrating that stored state can extend effective context without expanding the attention window.
arXiv:2511.03728v2 Announce Type: replace Abstract: On-device AI agents offer the potential for personalized, low-latency assistance, but their deployment is fundamentally constrained by limited memo...
arXiv:2602.22402v2 Announce Type: replace-cross Abstract: As large language models engage in extended reasoning tasks, they accumulate significant state -- architectural mappings, trade-off decisions...