LaCache introduces a redesigned semantic caching mechanism for large language model (LLM) serving that defends against cache‑collision attacks. By shifting integrity checks from queries to the first $k$ speculatively decoded tokens of the LLM response, LaCache can detect adversarial cache pollution with provable high probability. Experiments across multiple LLMs and benchmarks show that LaCache reduces attack hit rates to near zero while retaining over 90% of benign cache utility.
arXiv:2609.35908v1 Announce Type: cross Abstract: Semantic caches reduce LLM serving costs by reusing previously generated answers for semantically similar queries. However, retrieval is based solely...
arXiv:2601. 23088v2 Announce Type: replace-cross Abstract: Semantic caching has emerged as a pivotal technique for scaling LLM applications, widely adopted by major providers including AWS and Microsoft.
arXiv:2607. 19957v1 Announce Type: cross Abstract: Key-Value (KV) cache reduces inference latency in large language models (LLMs).
arXiv:2606. 02643v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG)-enhanced LLM systems, while powerful, introduce substantial inference costs due to the inclusion of an extra multi-stage pipeline that dynamically retrieves and synthesizes information from external knowledge sources.
The paper exposes a new vulnerability in Federated Retrieval-Augmented Generation (FedRAG) called Routing Hijacking, where a malicious client forges its semantic profile to attract target queries despite lacking relevant data. Experiments across three FedRAG routing architectures show that this attack consistently misroutes queries, causing downstream failures such as missing evidence, poisoning, incorrect answers, hallucinations, and sycophantic behavior. The authors propose a trust‑aware post‑routing framework that reweights clients based on evidence feedback, which effectively suppresses persistent hijacking and transfers to a learned neural router.
arXiv:2608. 28389v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) augments LLMs with external documents, but public or user-editable sources expose RAG systems to data poisoning: attackers can inject malicious documents to steer outputs toward targeted answers.
arXiv:2606. 12703v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) agents increasingly run with persistent memory that accumulates across user sessions.
arXiv:2608. 03844v1 Announce Type: new Abstract: Memory-augmented LLM agents rely on rich context for long-horizon reasoning and acting, yet their memory modules expose a persistent attack surface for malicious records, making the study of memory poisoning threats imperative.
arXiv:2607. 04281v1 Announce Type: cross Abstract: Semantic caching reduces the latency and cost of retrieval-augmented generation (RAG) by serving cached answers to semantically similar queries, but most existing methods do not model the time-varying freshness of open-web evidence.
The paper introduces an attack that reconstructs text generated by locally hosted large language models by monitoring CPU cache activity during detokenization. It uses Flush+Reload on shared tokenizer code to time decoding, then Prime+Probe to capture token‑dependent cache traces, followed by a clustering‑and‑language‑model pipeline to recover the output text. The method is evaluated across various datasets, hardware, inference frameworks, and model families, successfully retrieving semantically accurate outputs from real‑world local LLM deployments, including agentic systems.
The paper introduces two algorithms—CUCB‑SC‑MT and CLCB‑LS‑MT—for multi‑turn semantic caching in large language model serving. These methods learn a "reuse risk" metric from judge feedback to decide which conversation states to cache, balancing avoided generation cost against the risk of unacceptable reuse. Experiments on math‑tutoring and movie‑ticketing dialogues show that the online CLCB‑LS‑MT algorithm reduces generation cost by 35–41% and estimated latency by 19–35% compared to prior caching approaches.
arXiv:2406.00083v3 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by retrieving relevant information from external knowledge bases t...