arXiv:2608.23565v1 Announce Type: new Abstract: An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: contro...
The paper presents a memory layer called galahad-kv that stores the key-value state of a large language model’s context blocks (≈16,000 tokens each) on encrypted local NVMe disk, allowing byte‑exact retrieval without recomputation. On a single NVIDIA H100, the authors tested the system with Gemma 4 12B and 31B models over 50 million tokens of real public text, achieving 100 % successful reloads, 2.8×–4.3× faster loading than recomputation, and 8.8×–12.3× lower GPU energy usage while keeping GPU memory flat. The models answered long‑range factual queries with high accuracy (82 % for 12B, 98 % for 31B) and did not hallucinate, demonstrating that stored state can extend effective context without expanding the attention window.
The paper introduces the Compute-Value Audit (CVA), a sequential framework that evaluates whether extra sampling during test‑time scaling for video world models actually yields a net benefit after accounting for the compute cost of generation and verification. On 192 Physics‑IQ scenes, increasing the sample pool from 4 to 16 candidates improves oracle quality by +9.23 IQ, yet common metrics such as Flow, Cycle, and VideoReward fail to reliably recover this headroom, and adaptive‑depth policies recover only 42‑69% of the potential gain. Only a few specific interventions—anchor‑explorer in a sparse PRM800K setting, MMLU‑Pro exposing a predictive‑state gap, and a privileged paired‑future upper bound—successfully pass all CVA stages, indicating that sampling headroom is valuable only when it can be converted into a reliable decision that survives the full compute charge.
arXiv:2610.17434v1 Announce Type: cross Abstract: Recent advances in video world models have enabled plausible interactive environments, yet long-horizon generation places a fundamental demand on vis...
arXiv:2605. 09735v2 Announce Type: replace-cross Abstract: Static-graph LLM decoders provide predictable launches, fixed tensor shapes, and low submission overhead, but online decoding exposes highly irregular KV-cache behavior: request lengths differ, EOS events arrive asynchronously, and logical histories fragment over time.
arXiv:2604. 26968v2 Announce Type: replace-cross Abstract: Key-value (KV) cache memory management is the primary bottleneck limiting throughput and cost-efficiency in large-scale GPU inference serving.
arXiv:2610.19822v1 Announce Type: cross Abstract: LLM agents are evolving from a single model calling tools to planners that orchestrate multiple expert models whose capabilities live in distinct wei...
arXiv:2609.16215v1 Announce Type: new Abstract: GPU high bandwidth memory is scarce and expensive, and KV caches consume much of it as chats, agent loops, and document question answering accumulate s...
arXiv:2609.32540v2 Announce Type: replace-cross Abstract: Few-step autoregressive video diffusion generates a long video by splitting the video into temporal chunks and generating chunk-by-chunk, eac...
An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: control wants a short horizon, memory wants an unbounde...
arXiv:2608. 07408v1 Announce Type: cross Abstract: We study visual persistence in interactive video world models.
KVMem is a KV-context virtualization system that allows large language model agents to maintain workspaces exceeding both GPU key‑value capacity and the model’s native context window. It stores overflowed history as paged KV state across GPU memory, host memory, and NVMe, using lightweight, model‑native attention‑space indexes to retrieve relevant historical blocks. Evaluations on long‑context agent benchmarks show that KVMem improves task utility and inference efficiency, enabling up to one million‑token workspaces on consumer GPUs and achieving interactive responsiveness in local deployments.