The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
arXiv:2609.17109v1 Announce Type: new Abstract: A common small-model deployment runs one shared backbone with several LoRA specialists that answer over the same context. Serving them naively re-prefi...
arXiv:2608.30963v1 Announce Type: cross Abstract: Modern large language model (LLM) serving systems increasingly operate over repeated or shared context, yet each model typically performs its own pre...
The paper introduces KVLearn, a learning‑based framework for managing key‑value (KV) cache retention in disaggregated large language model (LLM) serving systems. KVLearn comprises a Prefix Reuse Predictor that estimates reuse probability from prefix features, a Cost‑Aware Retention Score that translates this probability into keep/admit decisions based on recomputation, transfer, and storage costs, and an Adaptive Threshold Controller that tunes admission thresholds online. Integrated into a globally disaggregated serving topology, KVLearn achieves up to 56% faster time‑to‑first‑token and up to 53% reduction in inter‑node KV transfer compared to baseline caching strategies.
arXiv:2610.02233v1 Announce Type: cross Abstract: Cross-context KV-cache reuse predicts a shared segment's keys and values under a new prefix instead of recomputing them, and has been reported to do...
arXiv:2608. 11231v1 Announce Type: new Abstract: LLM serving is increasingly accelerated by position-independent caching (PIC).
arXiv:2504. 15610v4 Announce Type: replace Abstract: Fine-tuning a 7B language model for specialized advising is attractive in resource-constrained settings, but multi-epoch runs routinely exceed the wall-clock limits of the free-tier GPUs (Kaggle, Colab) such users rely on.