arXiv Machine Learning September 17, 2026 By Saipraveen Vabbilisetty, Ajay Kumar Boddepalli, Deep Narayan Mishra, Shashank Kapadia, Haoan Wang, Anupriya Sharma

Beyond Static RAG: An Adaptive, Tri-Metric Routing Framework for Efficient Long-Context Inference on Commodity GPUs

Read the original on arXiv Machine Learning →

The Flow has not summarised this story yet — read it at arXiv Machine Learning.

arXiv AI

Aug 21

From Retrieved Context to Runtime Control: Adaptive Compression for Edge-based RAG

arXiv:2608. 19535v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) improves language-model responses by grounding generation in external passages, which comes with overhead: retrieved context lengthens the prompt, increasing prefill work, KV-cache footprint, memory traffic, latency, and energy.

By Zlatan Feric, Amir Taherin, Yanzhi Wang, David Kaeli

llms rag efficiency benchmarks

arXiv Machine Learning

Jul 30

InferScale: GPU-Native KV Injection for Personalized LLM Serving

arXiv:2607. 27090v1 Announce Type: cross Abstract: Large language models are increasingly deployed with persistent personalized context, such as accumulated memory profiles or long conversation histories, that is shared across a user's many requests.

By Peter Li, Prashant Pandey

llms rag fine-tuning efficiency

arXiv Machine Learning

Sep 10

MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference

arXiv:2609.07966v1 Announce Type: new Abstract: Key--value (KV) cache compression is an effective way to reduce the memory overhead of large language model (LLM) inference, particularly for long-cont...

By Michael Wang, Keith Li, Roozbeh Bostandoost

llms efficiency

arXiv AI

Jun 24

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

arXiv:2606. 24467v1 Announce Type: new Abstract: Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware.

By Xiaolin Lin, Jingcun Wang, Olga Kondrateva, Yiyu Shi, Bing Li, Grace Li Zhang

llms efficiency

arXiv AI

Sep 10

RAGMark: A Comprehensive Framework for Benchmarking Retrieval-Augmented Generation Systems

arXiv:2609.05760v1 Announce Type: cross Abstract: We present RAGMark, a modular benchmarking framework for advanced Retrieval-Augmented Generation (RAG) systems targeting small-scale multi-GPU enviro...

By Zlatan Feric, Amir Taherin, Bin Ren, Yanzhi Wang, Jennifer Dy, David Kaeli

rag efficiency benchmarks

Hugging Face Trending Papers

Jul 29

InferScale: GPU-Native KV Injection for Personalized LLM Serving

Large language models are increasingly deployed with persistent personalized context, such as accumulated memory profiles or long conversation histories, that is shared across a user's many requests. Production memory systems (e.

llms rag fine-tuning efficiency

One email a morning, machine-written

Related stories

From Retrieved Context to Runtime Control: Adaptive Compression for Edge-based RAG

InferScale: GPU-Native KV Injection for Personalized LLM Serving

MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

RAGMark: A Comprehensive Framework for Benchmarking Retrieval-Augmented Generation Systems

InferScale: GPU-Native KV Injection for Personalized LLM Serving