arXiv AI
Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Sep 15
Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs
A common small-model deployment runs one shared backbone with several LoRA specialists that answer over the same context. Serving them naively re-prefills that shared context once per specialist. We s...
arXiv AI
Sep 1
A Universal Context-Reuse Layer for Cross-Model KV Sharing
arXiv:2608.30963v1 Announce Type: cross Abstract: Modern large language model (LLM) serving systems increasingly operate over repeated or shared context, yet each model typically performs its own pre...
arXiv Machine Learning
Oct 22
To Keep or Not to Keep: Learning KV Cache Retention in Disaggregated LLM Serving Systems
The paper introduces KVLearn, a learning‑based framework for managing key‑value (KV) cache retention in disaggregated large language model (LLM) serving systems. KVLearn comprises a Prefix Reuse Predictor that estimates reuse probability from prefix features, a Cost‑Aware Retention Score that translates this probability into keep/admit decisions based on recomputation, transfer, and storage costs, and an Adaptive Threshold Controller that tunes admission thresholds online. Integrated into a globally disaggregated serving topology, KVLearn achieves up to 56% faster time‑to‑first‑token and up to 53% reduction in inter‑node KV transfer compared to baseline caching strategies.
arXiv Machine Learning
Oct 5
Budgeted Cache Repair for Cross-Context KV-Cache Reuse
arXiv:2610.02233v1 Announce Type: cross Abstract: Cross-context KV-cache reuse predicts a shared segment's keys and values under a new prefix instead of recomputing them, and has been reported to do...
arXiv AI
Oct 9
Freeze the Decoder, Heal the Encoder: Parameter-Efficient Adaptation for SVD-Based KV-Cache Compression
arXiv:2610. 10552v1 Announce Type: cross Abstract: Comparing parameter-efficient fine-tuning recipes under a single, shared learning rate is a common but flawed practice: when the arms being compared have very different trainable-parameter counts, a shared rate can simultaneously depress the larger arms' means and inflate their variance, manufacturing a large, seemingly multi-seed-significant advantage for the smallest arm that is not a real effect.
arXiv Machine Learning
Aug 4
RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
arXiv:2608. 01247v1 Announce Type: cross Abstract: Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets.