arXiv AI By Shuncheng Jie, Desen Sun, Ri Hong, Yuyang Tian, Hong Zhang, Sihang Liu

SAKURA: Serving Attention Key-Value State Under Rate Adaptation for Private Image Editing on Phones

Read the original on arXiv AI →

SAKURA is a cloud-edge system that delivers reusable attention key‑value state to mobile devices for diffusion‑based image editing while respecting a bandwidth budget. It compresses the state with a token‑importance‑aware method, dynamically selects cache representations based on network conditions, and overlaps transfer with on‑device computation. Experiments on a Snapdragon 8 Elite Gen 5 phone show that SAKURA cuts latency by up to 3.27× and reduces cache storage by 3.06× compared to uncompressed caches, all while maintaining image quality.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Jun 11

MiniPIC: Flexible Position-Independent Caching in <100LOC

Retrieval-augmented and agentic workloads repeatedly prefill recurring predictable structured inputs (which we call "spans") such as documents and code files. Yet, prefix caching in engines such as vLLM cannot reuse their KV entries unless they share identical prefixes with another request, while Position-Independent Caching (PIC) implementations within production-grade inference servers typically either require substantial server code changes or keep KV state outside the server, incurring host-to-device transfer overhead.