arXiv AI
Jul 22
M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generation
arXiv:2603. 26667v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) turns external documents into evidence for large language models.
REVA: Reusable Evidence View Aggregation for Context-Efficient RAG Serving
The paper introduces REVA, a method for compressing retrieval-augmented generation (RAG) prompts by aggregating historical query–document–model interactions into reusable evidence views. REVA mines attention traces from the target generator, maps token-level attention to readable words, aggregates importance across repeated document accesses, and produces budget‑specific plain‑text views that maintain document order and the standard RAG interface. Experiments on four benchmarks with modern LLMs show that REVA improves generation quality by 1.0–5.8 points over existing compressors while reducing compression overhead by 5.3 to 15.6 times and adding less than 40 ms of latency.
Related stories
arXiv AI
Aug 25
MegaMem: A Retrieval Solution for Ultra-Large Context Windows
arXiv:2608.22137v1 Announce Type: new Abstract: Modern language models and agents increasingly require persistent memory for complete codebases, long interaction histories, and heterogeneous enterpri...
arXiv AI
Jul 21
SALT: Salience-Aware Lexical Trie for Long-Context Compression
arXiv:2607. 17486v1 Announce Type: cross Abstract: As large language models (LLMs) process increasingly longer prompts, computation and KV-cache memory costs have emerged as major bottlenecks in inference systems.
Hugging Face Trending Papers
Jun 4
QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving
Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost. RAG cache fusion reduces this cost by reusing precomputed key-value (KV) caches for retrieved chunks and selectively recomputing tokens under the current prompt.
arXiv AI
Aug 20
ICICLE: Expanding Retrieval with In-Context Documents
ICICLE is an in‑context indexing framework that expands generative retrieval by supplying newly added documents as inference‑time evidence. It generates document identifiers using both parametric memory and context‑provided document‑docid pairs, employing a [COPY] routing mechanism, preference‑based calibration, and large‑context adaptation to separate context‑grounded retrieval from parametric retrieval. Experiments on MS MARCO and NQ320K demonstrate that ICICLE improves retrieval of new documents while retaining performance on previously indexed documents without retraining the model.
arXiv AI
Aug 19
DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval
The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.
arXiv AI
Jun 6
QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving
arXiv:2606. 05875v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost.
arXiv AI
Jul 21
SelKV: Selective KV Cache Merging with Per-Token Merge-or-Drop and Attention Compensation
arXiv:2607. 16213v1 Announce Type: new Abstract: Large Language Models (LLMs) generate text autoregressively, relying on a key-value (KV) cache whose memory footprint grows linearly with context length, creating a major bottleneck.
arXiv AI
Jun 4
MemoryDocDataSet: A Benchmark for Joint Conversational Memory and Long Document Reasoning
arXiv:2606. 04442v1 Announce Type: cross Abstract: AI systems increasingly need to combine two demanding capabilities: navigating multi-session conversation history and performing deep reading comprehension within long documents.
arXiv Machine Learning
Jul 1
InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context
arXiv:2603. 05353v2 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts.
arXiv AI
Jul 21
Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restoration
arXiv:2607. 16848v1 Announce Type: cross Abstract: Long-term memory is becoming a core component of LLM agents, but most memory benchmarks evaluate conversations or compact summaries, while research agents need to restore evidence from full scientific papers.
arXiv AI
Aug 18
What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering
arXiv:2608. 14841v1 Announce Type: new Abstract: Long-document visual question answering (VQA) over documents of tens to hundreds of pages mixing text, tables, charts, and figures typically follows retrieve-then-read pipelines.