arXiv Machine Learning
AdaMem: Adaptive Memory Token Allocation for Soft Compression in Retrieval-Augmented Generation
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
Sep 7
Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG
The paper introduces DEX-Comp, a two‑stage training method for soft context compression in Retrieval‑Augmented Generation (RAG). First, a pure distillation warm‑start trains the compression model on correct responses from an uncompressed RAG. Then, hard exploration uses reinforcement learning on queries where the uncompressed RAG fails, encouraging better computation patterns for compressed representations. Experiments on five open‑domain QA benchmarks show that DEX‑Comp compresses retrieved contexts 16×, speeds inference 4×–24×, and matches or surpasses the uncompressed RAG baseline across various retrieval depths.
arXiv AI
Aug 19
DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval
The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.
arXiv AI
Jul 21
SelKV: Selective KV Cache Merging with Per-Token Merge-or-Drop and Attention Compensation
arXiv:2607. 16213v1 Announce Type: new Abstract: Large Language Models (LLMs) generate text autoregressively, relying on a key-value (KV) cache whose memory footprint grows linearly with context length, creating a major bottleneck.
arXiv Machine Learning
Jul 20
VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs
arXiv:2607. 15498v1 Announce Type: cross Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference.
arXiv Computation and Language
Aug 25
SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
arXiv:2606.16661v2 Announce Type: replace-cross Abstract: Fixed-length chunking in Retrieval-Augmented Generation (RAG) often leads to boundary fragmentation, where critical evidence is split across...
arXiv Computation and Language
2d ago
Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
Adamas is a token‑level sparse attention mechanism that uses the Hadamard transform, bucketization, and 2‑bit compression to create compact representations, followed by lightweight distance‑based top‑k selection for efficient long‑context inference. It achieves near‑lossless performance on Llama‑3.1‑8B‑Instruct and delivers up to 6.5× speedup in self‑attention and 1.7× in end‑to‑end decoding at a 32K context length. Adamas also matches full attention performance on challenging long‑form reasoning benchmarks such as AIME 2024 and 2025.