The paper "Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Model" argues that traditional algorithmic latency attacks are largely ineffective against modern LLM serving systems due to system-level optimizations like continuous batching. It introduces a new Fill and Squeeze attack that targets the scheduler’s state transition, exhausting the global KV cache to cause Head-of-Line blocking and forcing repetitive preemption. Experiments on vLLM show the attack can cause up to 75–742× TTFT degradation and 1.5–4× slowdown on Time Per Output Token, achieving these effects with 30–40% lower attack cost.
arXiv:2601. 00389v2 Announce Type: replace-cross Abstract: Timing and burst patterns can leak through encryption, and an adaptive adversary can exploit them.
The paper investigates how multi‑step tool‑calling large language model agents can unintentionally create persistent billable state, allowing malicious or compromised tools to generate repeated charges without user credentials. It formalizes the persistent billable‑state boundary, identifies six denial‑of‑wallet attack vectors, and evaluates them with the DOW‑BENCH harness across six model families. The study shows significant cost amplification, demonstrates effective mitigation via deterministic history transformation and host‑side invariants, and highlights the scarcity of existing safeguards in real‑world repositories.
arXiv:2604. 17249v2 Announce Type: replace-cross Abstract: Rowhammer on GPU DRAM has enabled adversarial bit flips in model weights; shared KV-cache blocks in LLM serving systems present an analogous but previously unexamined target.
arXiv:2609.22818v1 Announce Type: cross Abstract: Memory-poisoning defenses for LLM agents are typically evaluated by their ability to prevent attacks. However, the traffic they process is rarely adv...
arXiv:2608. 08382v1 Announce Type: new Abstract: As LLM inference shifts to multi-tenant GPU clusters, co-batching improves throughput but obscures per-tenant usage and limits control.
arXiv:2610.28425v1 Announce Type: cross Abstract: Modern LLM agents persist state across sessions via two channels: long-lived memory files silently prepended to the system prompt, and scheduled task...
arXiv:2607. 02043v1 Announce Type: cross Abstract: Disaggregated LLM serving runs prefill and decode on separate GPU pools to keep the two phases from interfering.
arXiv:2605. 08876v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly deployed as autonomous agents that execute tool-augmented, multi-step tasks, where latency is a critical factor for real-world applications.
arXiv:2608. 09225v1 Announce Type: cross Abstract: The key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference, enabling prefix reuse across requests.
arXiv:2607. 24692v1 Announce Type: cross Abstract: Inference systems increasingly combine a fast path that returns predictions within the application's latency deadline together with a higher-accuracy slow path that runs higher-compute methods on stronger, remote hardware, so its results can be returned on time and combined with the fast path predictions.
The paper introduces FairInference, a system that guarantees token-level latency isolation for well-behaved clients in multi-tenant LLM serving. It provides a δ-token fairness guarantee, ensuring that a token generated in isolation within time d will be produced within d + δ in a shared environment. The approach enforces per-token deadlines, bounds GPU compute sharing delays, and accounts for shared KV cache overhead, leading to reduced latency spikes and higher overall throughput compared to existing LLM serving systems.
The paper introduces an elastic key‑value (KV) cache for large language model (LLM) serving that dynamically reclaims a pre‑allocated reserve during decode‑heavy phases and restores it before prefill, using a userspace CUDA virtual‑memory trick that requires no driver changes. The authors implement this mechanism, test it under realistic workloads, and find that it offers only marginal benefits—about a 1 % difference in time‑to‑first‑token for large prefill chunks—and that simpler strategies such as lowering the maximum batch size can achieve similar results. The study also notes that the reserve’s impact diminishes with higher tensor‑parallelism levels. whyItMatters":"The work demonstrates that a dynamic KV cache reclamation strategy can be implemented without driver patches and that its practical benefits are limited, guiding future LLM serving optimizations toward simpler approaches."