The paper evaluates how different chunking strategies—fixed-size, recursive paragraph-first, and content-defined—affect the cost of updating embeddings when Wikipedia documents are revised. Using 120 pairs of Chinese and English revisions, the authors find that fixed-size chunking requires the most recomputation (57‑68 % of the workload), while recursive and content-defined chunking need less (38‑56 % and 43‑54 % respectively). Despite these differences, evidence recall under a 2048‑token retrieval budget varies by less than two percentage points across methods and languages.
arXiv:2606.16661v2 Announce Type: replace-cross Abstract: Fixed-length chunking in Retrieval-Augmented Generation (RAG) often leads to boundary fragmentation, where critical evidence is split across...
The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.
arXiv:2609.24220v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presen...
The paper argues that retrieval‑augmented question‑answering systems should perform semantic compilation at ingest time rather than re‑deriving meaning at query time. By building a maintained structure—incrementally updated embeddings and validated atomic claims—read operations become far cheaper, with experimental results showing higher accuracy and lower token usage compared to traditional chunk‑based retrieval. The authors present two proofs: cheaper incremental updates and superior performance on broadcast‑interview transcripts, suggesting a new systems agenda for compilation and read planning.
arXiv:2607. 24767v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems have emerged as a powerful process for allowing large language models (LLMs) to retrieve relevant information to use as source material during text generation.
arXiv:2608. 04569v1 Announce Type: cross Abstract: Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget.
arXiv:2610.17338v1 Announce Type: new Abstract: Language-model agents revise their working context as information becomes obsolete, but deleting text does not erase its influence on cached computatio...
PRIME introduces an index-side optimization framework for structured document retrieval, comprising a Semantic Restructuring track that converts long documents into dense, structured sections and a Metadata Synthesis & Hierarchical Retrieval track that extracts semantic anchors and heading structures for efficient coarse-to-fine retrieval. The approach reduces embedding computation by 45.8× and index vector count by 22.4× while maintaining retrieval quality comparable to structure-aware chunking. In full deployment, Semantic Restructuring improves Answer‑F1 by 13.7%, and the study identifies qualitative regimes that influence when index-side optimization is beneficial or detrimental, noting that strong rerankers neutralize these effects.
arXiv:2601. 00821v3 Announce Type: replace Abstract: A growing class of conversational-memory systems compresses dialogue history into structured artifacts -- extracted facts, decisions, or events -- on the premise that distilled structure retrieves better than raw text.
arXiv:2608. 06305v1 Announce Type: new Abstract: Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query.
arXiv:2608.29899v1 Announce Type: cross Abstract: Dense retrieval over long documents is expensive. Token-level encoders scale quadratically in sequence length, and most long-context embedding models...
KV-Repurpose is an open‑source, vLLM‑based system that enables segment‑wise KV‑cache reuse beyond exact prompt prefixes in large language model serving. It integrates cache‑aware segment scheduling, prompt construction, capacity management, workload generation, and runtime monitoring to evaluate non‑prefix KV‑cache reuse. Experiments show that naive segment reuse maintains output quality for single‑document QA and summarization, while significantly improving cache hit rates and reducing time‑to‑first‑token compared to prefix caching, especially under Zipfian RAG‑style workloads.