arXiv Machine Learning August 4, 2026 By Anusha Madan Gopal, Aras Pirbadian, Kristofor D. Carlson, M Anthony Lewis, Jonathan Tapson

Structured Memory for Edge Language Models: Persistent Context and Corpus Retrieval via O(1) SSM State Injection

Read the original on arXiv Machine Learning →

arXiv:2608. 02560v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) imposes a prefill cost proportional to retrieved context length, and -- with Transformer backbones -- a KV-cache that grows with each generated token.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 30

InferScale: GPU-Native KV Injection for Personalized LLM Serving

arXiv:2607. 27090v1 Announce Type: cross Abstract: Large language models are increasingly deployed with persistent personalized context, such as accumulated memory profiles or long conversation histories, that is shared across a user's many requests.

By Peter Li, Prashant Pandey

llms rag fine-tuning efficiency

Hugging Face Trending Papers

Jul 29

InferScale: GPU-Native KV Injection for Personalized LLM Serving

Large language models are increasingly deployed with persistent personalized context, such as accumulated memory profiles or long conversation histories, that is shared across a user's many requests. Production memory systems (e.

llms rag fine-tuning efficiency

arXiv Computation and Language

Sep 14

SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering

arXiv:2608.00311v2 Announce Type: replace Abstract: Long-context inference with large language models (LLMs) is costly: self-attention during prefill scales quadratically with sequence length, and th...

By Maryam Haghifam, Jason Cong, Yizhou Sun

llms nlp fine-tuning efficiency benchmarks

arXiv Computation and Language

3d ago

The Evolution of Attention in Large Language Models: Mechanisms, Trade-offs, and Emerging Trends

arXiv:2609.39661v1 Announce Type: new Abstract: Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value c...

By Zhentao Tan, Jingyi Shen, Yanbo Li, Yao Liu, Yue Wu, Jieping Ye

llms

arXiv Machine Learning

Jul 9

TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models

arXiv:2607. 07388v1 Announce Type: cross Abstract: Large Language Models (LLMs) store factual knowledge and domain-specific patterns implicitly in dense Transformer parameters, making knowledge expansion costly through pretraining, fine-tuning, retrieval augmentation, or longer contexts.

By Yutang Ma, Kecheng Huang, Xikun Jiang, Zili Shao

llms fine-tuning efficiency

arXiv AI

Jun 24

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

arXiv:2606. 24467v1 Announce Type: new Abstract: Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware.

By Xiaolin Lin, Jingcun Wang, Olga Kondrateva, Yiyu Shi, Bing Li, Grace Li Zhang

llms efficiency

One email a morning, machine-written

Related stories

InferScale: GPU-Native KV Injection for Personalized LLM Serving

InferScale: GPU-Native KV Injection for Personalized LLM Serving

SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering

The Evolution of Attention in Large Language Models: Mechanisms, Trade-offs, and Emerging Trends

TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference