CliffCompaction is an autocompaction technique that reduces cost by up to 50% while maintaining or improving performance on benchmarks such as Terminal‑Bench and KernelBench. It achieves this by truncating or dropping content without rephrasing, ensuring compacted information remains faithful and preventing context drift. The method enables efficient test‑time scaling, matching or surpassing higher‑cost models like Opus 4.7 and GPT‑5.3 Codex, and delivers significant CUDA kernel speedups on KernelBench.
arXiv:2607. 08032v1 Announce Type: new Abstract: Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions.
SCLATE is a new execution substrate that allows continual‑learning benchmarks and agents to share a single event scheduler via adapters, enabling tasks, session events, and memory consolidation to run on a compressed, real‑time timeline. It also functions as a rollout engine that records every model call’s tokens and log probabilities without modifying the agent’s harness or memory. Using SCLATE, the authors ported seven benchmarks, compared ten harness‑memory configurations across ten models, and demonstrated that post‑training Qwen3.5‑4B can effectively leverage both harness and memory, improving performance on multiple metrics.
KVMem is a KV-context virtualization system that allows large language model agents to maintain workspaces exceeding both GPU key‑value capacity and the model’s native context window. It stores overflowed history as paged KV state across GPU memory, host memory, and NVMe, using lightweight, model‑native attention‑space indexes to retrieve relevant historical blocks. Evaluations on long‑context agent benchmarks show that KVMem improves task utility and inference efficiency, enabling up to one million‑token workspaces on consumer GPUs and achieving interactive responsiveness in local deployments.
arXiv:2608. 11242v1 Announce Type: cross Abstract: When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks.
Modern LLM coding agents such as Claude Code and OpenHands share a common inefficiency: they spend much of their token budget finding the file to patch, rather than patching it. On SWE-Bench Verified, a 30B OpenHands agent averages 23 rounds and 631K tokens per resolved issue, with many calls spent on grep, glob, and view_file during repository exploration.
arXiv:2607. 01523v1 Announce Type: cross Abstract: Recurrent memory agents extend LLMs to arbitrarily long contexts by iteratively consolidating input into a fixed-size memory window.
The paper introduces KOPE, an experience‑driven framework that records hardware kernel optimization trajectories in an Experience Graph Memory and uses Active Context Management and Injection to retrieve relevant past decisions under a fixed token budget. KOPE preserves decision order, outcomes, and alternative branches, enabling evidence from completed runs to inform future optimization steps. In experiments, KOPE achieves a 1.54× speedup over the strongest baseline, raises pass rates from 60.0% to 84.6%, and reduces token consumption dramatically, demonstrating the benefits of continual learning from external experience while keeping the foundation model unchanged.
Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware. Existing KV cache eviction methods typically apply heuristic token scoring over all heads in GQA-based LLMs.
arXiv:2606. 09659v1 Announce Type: cross Abstract: Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.
arXiv:2608. 05886v1 Announce Type: cross Abstract: Modern LLM coding agents such as Claude Code and OpenHands share a common inefficiency: they spend much of their token budget finding the file to patch, rather than patching it.
arXiv:2606. 26666v1 Announce Type: new Abstract: Autoregressive large language model (LLM) serving is increasingly limited by key-value (KV) cache movement rather than dense matrix multiplication.
Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length. Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long prompt.