arXiv AI By Yang Liu, Zhaokai Luo, Huayi Jin, Ruozhou He, Chenchen Hong, Zhiyong Wang, Yifei Liu, Yunfei Gu, Chentao Wu, Junhao Hu

Akashic: A Low-Overhead LLM Inference Service with MemAttention

Read the original on arXiv AI →

arXiv:2607. 05708v1 Announce Type: new Abstract: Recent LLM-based agent systems continuously accumulate context across multi-turn interactions, tool invocations, and cross-session workflows.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Jul 7

Akashic: A Low-Overhead LLM Inference Service with MemAttention

Recent LLM-based agent systems continuously accumulate context across multi-turn interactions, tool invocations, and cross-session workflows. Replaying the full history for every request quickly becomes impractical: long contexts increase prefill cost, may exceed context limits, and often bury task-relevant evidence in irrelevant content, degrading both serving efficiency and output quality.

arXiv AI
Sep 24

Memory Control Signals Emerge Before Action in Long Horizon Agents

The paper investigates how long‑horizon language model agents encode memory‑management signals before taking actions. By examining hidden states just prior to each action, the authors find that the model already signals the need for compression and recall, independent of context length or interaction progress, and that these signals vary across model depth. They propose the Preaction Memory with Evidence Retrieval (PaMER) framework, which uses state‑guided compression and selective evidence retrieval to reduce context consumption while preserving task performance.

By Mingxuan Wang, Guorun Yao, Fei Luo, Yinglong Guo, Chao Ning, Bo Wang, Hongyue Chen, Yanbiao Ma, Jungong Han