Hugging Face Blog
Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval
Read the original on Hugging Face Blog →The Flow has not summarised this story yet — read it at Hugging Face Blog.
Related stories
arXiv AI
Jun 9
Projection and Quantisation: A Unifying View of Learning to Hash, from Random Projections to the RAG Era
arXiv:2510. 04127v2 Announce Type: replace-cross Abstract: Approximate nearest neighbour (ANN) search underpins large-scale retrieval, increasingly within the retrieval-augmented generation pipelines that ground large language models, yet the methods that address it have multiplied across communities until they are seldom read as a single field.
arXiv AI
Sep 10
Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval
Matryoshka Hash Representations (MHR) propose a two‑stage quantization approach for retrieval‑augmented generation. First, a long binary code is learned; then, frozen, additional zero‑initialized residual adaptors are trained to produce searchable prefixes of varying byte budgets. Evaluated on MS MARCO and transferred to seven BEIR datasets, MHR achieves higher NDCG@10 and Recall@100 at 32‑byte budgets than baselines, especially in low‑budget regimes, and can also improve candidate shortlisting and graph‑index pruning.
arXiv AI
Sep 17
Quanta: A Self-Contained Python Library for Hybrid Retrieval over Quantised Embeddings, Lexical Indexes, and Knowledge Graphs
Quanta is an open‑source Python library that unifies dense vector search over 4‑bit quantised embeddings, BM25 full‑text retrieval, and knowledge‑graph traversal behind a single retrieval API. It combines signals using weighted reciprocal rank fusion instead of normalising heterogeneous scores, arguing that such normalisations are query‑dependent. The library treats the graph as a candidate expander rather than a relevance scorer, widening the candidate pool and then re‑scoring documents with dense indexes under an identifier allowlist.
arXiv AI
Jun 11
What Limits Does Quantization Place on Dense Top-$k$ Retrieval? A Theoretical Study
arXiv:2606. 11780v1 Announce Type: cross Abstract: We establish conditions for embedding a corpus of $N$ documents as $d$-dimensional vectors such that every $k$-subset $S \subseteq [N]$ is realizable as a result of top-$k$ retrieval by some query vector.
arXiv AI
Oct 5
MRVQ: One Resident Index for Dimension- and Rate-Elastic Vector Search
MRVQ (Matryoshka Residual Vector Quantization) is a single post‑hoc residual quantizer designed for frozen embeddings that can be truncated to support any combination of embedding dimension and code rate. By dropping residual stages or embedding coordinates, MRVQ provides a single resident artifact that serves all evaluated (dimension, rate) pairs, achieving the lowest memory usage among tested designs. Although it uses 17.8–22.0× less memory than three separately trained QINCo2 indices and 1.89–2.02× less than a lean shared‑model steelman, it trades off a small drop in nDCG@10 on FiQA compared to per‑rate QINCo2, yet still outperforms PQ, OPQ, and AdANNS‑OPQ at matched code size.
arXiv AI
Sep 30
Quantization Enables Private Dense Retrieval against Malicious Service Providers
arXiv:2609.36376v1 Announce Type: cross Abstract: Dense retrieval, the key component of Retrieval Augmented Generation (RAG), retrieves the most relevant documents by comparing dense vector represent...