arXiv AI
Attention Calibration for Position-Fair Dense Retrieval
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 3
Attention Calibration for Position-Fair Dense Information Retrieval
arXiv:2606. 02737v1 Announce Type: cross Abstract: Dense retrieval models exhibit positional bias: retrieval effectiveness degrades when relevant information appears later in a passage (Zeng et al.
arXiv AI
Oct 21
Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity
The paper introduces Stratified Sparse Attention (SSA), a framework that blends deterministic top‑k retrieval with score‑guided sampling to reduce the gap between sparse and dense attention in large language models. SSA attends to high‑scoring tokens deterministically while sampling the remaining attention mass using proxy scores, thereby recovering more of the attention distribution. Experiments on benchmarks such as RULER, LongBench, LOFT, and AIME show that SSA outperforms existing sparse methods across models ranging from 3B to 27B parameters, achieving significant accuracy gains and up to 2.96× decode speedup with optimized CUDA kernels.
arXiv Machine Learning
Sep 22
AdaMem: Adaptive Memory Token Allocation for Soft Compression in Retrieval-Augmented Generation
arXiv:2609.22100v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves language models with retrieved evidence, but processing many long passages is costly and can introduce...
arXiv Machine Learning
Jul 31
Recall Before You Rank: Similarity-Guided Top-$K$ Reuse for Efficient Long-Context Attention
arXiv:2607. 27692v1 Announce Type: cross Abstract: Top-$K$ sparse attention reduces the cost of Softmax and value aggregation by attending to only a small subset of key--value (KV) entries.
arXiv AI
Aug 19
DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval
The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.
arXiv Machine Learning
Jun 4
LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding
arXiv:2606. 04302v1 Announce Type: cross Abstract: Key-value (KV) caching accelerates inference of large language models (LLMs) by reusing past computations for generated tokens.