← Back to all news
arXiv Computation and Language By Shuyu Guo, Shuo Zhang, Zhaochun Ren

Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG

Read the original on arXiv Computation and Language →

The paper introduces DEX-Comp, a two‑stage training method for soft context compression in Retrieval‑Augmented Generation (RAG). First, a pure distillation warm‑start trains the compression model on correct responses from an uncompressed RAG. Then, hard exploration uses reinforcement learning on queries where the uncompressed RAG fails, encouraging better computation patterns for compressed representations. Experiments on five open‑domain QA benchmarks show that DEX‑Comp compresses retrieved contexts 16×, speeds inference 4×–24×, and matches or surpasses the uncompressed RAG baseline across various retrieval depths.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Jun 9

End-to-End Context Compression at Scale

arXiv:2606. 09659v1 Announce Type: cross Abstract: Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.

By Ang Li, Sean McLeish, Haozhe Chen, Nimit Kalra, Zaiqian Chen, Artem Gazizov, Venkata Anoop Suhas Kumar Morisetty, Bhavya Kailkhura, Harshitha Menon, Zhuang Liu, Brian R. Bartoldson, Tom Goldstein, Sanae Lotfi, Micah Goldblum, Pavel Izmailov
llmsragagentsefficiency
More like this →
arXiv AI
Aug 19

DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval

The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.

By Jingyuan Wang, Richong Zhang, Zhijie Nie, Mingxin Li, Yanzhao Zhang
llmsragbenchmarks
More like this →
Hugging Face Trending Papers
Jun 9

One Token per Multimodal Evidence: Latent Memory for Resource-Constrained QA

External memory effectively grounds large language models (LLMs) and vision-language models (VLMs)-based question answering (QA) in relevant multimodal evidence. However, existing memory paradigms represent each memory item in raw text and image forms, so retrieval-based systems must pass the retrieved text or images to the generation LLMs/VLMs, resulting in high token consumption and storage pressure, making it unaffordable for resource-constrained applications.

llmsragnlpefficiencymultimodalbenchmarks
More like this →