arXiv Machine Learning
AgentKV: Phase-Aware KV Eviction for Agentic LLMs
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Aug 18
Learning Agent Execution for KV-Cache Management in Agentic Serving
arXiv:2608. 14624v1 Announce Type: new Abstract: Multi-agent LLM systems have emerged as an important deployment paradigm for AI services, where each user request is decomposed into a sequence of specialized agents.
arXiv Computation and Language
Aug 21
ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents
arXiv:2608. 19662v1 Announce Type: new Abstract: Agentic language models repeatedly encode tool and skill schemas that recur across requests in different combinations and orders, preventing standard prefix caching from reusing their key--value (KV) states.
arXiv AI
Jun 10
IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference
arXiv:2606. 09916v1 Announce Type: cross Abstract: Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and intermediate reasoning.
arXiv Machine Learning
Jun 29
Learning to Evict from Key-Value Cache
arXiv:2602. 10238v2 Announce Type: replace-cross Abstract: The growing size of Large Language Models (LLMs) makes efficient inference challenging, primarily due to the memory demands of the autoregressive Key-Value (KV) cache.
arXiv AI
3d ago
ActKV: Efficient LLM Agents through Action-Guided KV Cache Management
ActKV is a new KV cache compression framework designed for agentic large language model (LLM) inference. It prioritizes cache entries that contribute to action generation, using action-oriented eviction, confidence-driven budget allocation, and page-aware compression to reduce memory usage while preserving accuracy. In long-trace tasks, ActKV retains 98.53% of FullKV’s accuracy using only 25.98% of its peak memory and boosts token and task throughput by 3.97× and 3.58×, respectively.
arXiv Machine Learning
Jun 2
LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents
arXiv:2602. 01053v2 Announce Type: replace Abstract: Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents share a pretrained backbone and differ only by lightweight adapters.