arXiv AI By Yin Lin, Elaine Ang, Erkang Zhu, Bolin Ding, Jingren Zhou

Context as an Environment: Programmatic Context Management for Long-Horizon Agents

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv Machine Learning
Sep 7

KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU

KVMem is a KV-context virtualization system that allows large language model agents to maintain workspaces exceeding both GPU key‑value capacity and the model’s native context window. It stores overflowed history as paged KV state across GPU memory, host memory, and NVMe, using lightweight, model‑native attention‑space indexes to retrieve relevant historical blocks. Evaluations on long‑context agent benchmarks show that KVMem improves task utility and inference efficiency, enabling up to one million‑token workspaces on consumer GPUs and achieving interactive responsiveness in local deployments.

By Di Chai, Leye Wang, Zeshen Su, Zhiguo Xia, Zhihang Yu