Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG
Read the original on arXiv Computation and Language →The paper introduces DEX-Comp, a two‑stage training method for soft context compression in Retrieval‑Augmented Generation (RAG). First, a pure distillation warm‑start trains the compression model on correct responses from an uncompressed RAG. Then, hard exploration uses reinforcement learning on queries where the uncompressed RAG fails, encouraging better computation patterns for compressed representations. Experiments on five open‑domain QA benchmarks show that DEX‑Comp compresses retrieved contexts 16×, speeds inference 4×–24×, and matches or surpasses the uncompressed RAG baseline across various retrieval depths.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.