arXiv AI
Jul 22
M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generation
arXiv:2603. 26667v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) turns external documents into evidence for large language models.
REVA: Reusable Evidence View Aggregation for Context-Efficient RAG Serving
The paper introduces REVA, a method for compressing retrieval-augmented generation (RAG) prompts by aggregating historical query–document–model interactions into reusable evidence views. REVA mines attention traces from the target generator, maps token-level attention to readable words, aggregates importance across repeated document accesses, and produces budget‑specific plain‑text views that maintain document order and the standard RAG interface. Experiments on four benchmarks with modern LLMs show that REVA improves generation quality by 1.0–5.8 points over existing compressors while reducing compression overhead by 5.3 to 15.6 times and adding less than 40 ms of latency.
Related stories
arXiv AI
Aug 25
MegaMem: A Retrieval Solution for Ultra-Large Context Windows
arXiv:2608.22137v1 Announce Type: new Abstract: Modern language models and agents increasingly require persistent memory for complete codebases, long interaction histories, and heterogeneous enterpri...
arXiv AI
Jul 21
SALT: Salience-Aware Lexical Trie for Long-Context Compression
arXiv:2607. 17486v1 Announce Type: cross Abstract: As large language models (LLMs) process increasingly longer prompts, computation and KV-cache memory costs have emerged as major bottlenecks in inference systems.
Hugging Face Trending Papers
Jun 4
QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving
Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost. RAG cache fusion reduces this cost by reusing precomputed key-value (KV) caches for retrieved chunks and selectively recomputing tokens under the current prompt.
arXiv AI
Aug 20
ICICLE: Expanding Retrieval with In-Context Documents
ICICLE is an in‑context indexing framework that expands generative retrieval by supplying newly added documents as inference‑time evidence. It generates document identifiers using both parametric memory and context‑provided document‑docid pairs, employing a [COPY] routing mechanism, preference‑based calibration, and large‑context adaptation to separate context‑grounded retrieval from parametric retrieval. Experiments on MS MARCO and NQ320K demonstrate that ICICLE improves retrieval of new documents while retaining performance on previously indexed documents without retraining the model.
arXiv Machine Learning
Sep 22
AdaMem: Adaptive Memory Token Allocation for Soft Compression in Retrieval-Augmented Generation
arXiv:2609.22100v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves language models with retrieved evidence, but processing many long passages is costly and can introduce...
arXiv AI
Sep 28
Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding
The paper introduces Highlight-Then-Summarize (H2S), a two-step approach that first highlights question-relevant evidence in long documents and then condenses it into a compact, question-conditioned summary before generating an answer. The authors built the H2S-Dataset with 6,647 examples spanning 11 benchmark families, and developed H2S-RL to reward evidence selection and summary construction. Evaluated on the H2S-Bench suite, the H2S-14B model outperforms larger open-source models, achieving the highest overall score and maintaining strong performance even with a reduced output budget.
arXiv AI
Aug 19
DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval
The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.
arXiv AI
Sep 16
ORDER: Task-Conditioned Routing for Retrieval-Augmented Generation
The paper introduces ORDER, a task‑conditioned retrieval‑augmented generation framework that dynamically adapts both indexing and retrieval strategies to each incoming query. It first clusters questions to learn cluster‑specific chunking, metadata filtering, and reranking settings, then routes queries to the appropriate pre‑built index via nearest‑centroid assignment. Additionally, a supervised query router predicts relevant collections and a Uniform Multi‑source Sampler distributes the retrieval budget evenly across selected sources, yielding superior performance on heterogeneous historical archives compared to existing RAG systems.
arXiv Computation and Language
Oct 8
Document Optimization for Black-Box Retrieval via Reinforcement Learning
arXiv:2604.05087v4 Announce Type: replace Abstract: Generative large language models (LLMs) are increasingly used as inference-time components in retrieval pipelines, for tasks such as query rewritin...
arXiv AI
Jun 6
QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving
arXiv:2606. 05875v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost.
arXiv AI
Oct 5
Evaluating the Retrieval Robustness of Large Language Models
The paper evaluates how robust large language models (LLMs) are when using retrieval‑augmented generation (RAG) in practical settings. It investigates whether RAG always outperforms non‑RAG approaches, whether adding more retrieved documents helps, and whether the order of documents matters, using a benchmark of 1,891 samples across five datasets and three task categories. Experiments with 11 LLMs show generally high retrieval robustness, but performance varies by task and prompting strategy, indicating that adopting RAG should be considered on a case‑by‑case basis.