← Back to all news
arXiv AI By Peichun Hua, Yunming Xiao

Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval

Read the original on arXiv AI →

Matryoshka Hash Representations (MHR) propose a two‑stage quantization approach for retrieval‑augmented generation. First, a long binary code is learned; then, frozen, additional zero‑initialized residual adaptors are trained to produce searchable prefixes of varying byte budgets. Evaluated on MS MARCO and transferred to seven BEIR datasets, MHR achieves higher NDCG@10 and Recall@100 at 32‑byte budgets than baselines, especially in low‑budget regimes, and can also improve candidate shortlisting and graph‑index pruning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jun 9

Projection and Quantisation: A Unifying View of Learning to Hash, from Random Projections to the RAG Era

arXiv:2510. 04127v2 Announce Type: replace-cross Abstract: Approximate nearest neighbour (ANN) search underpins large-scale retrieval, increasingly within the retrieval-augmented generation pipelines that ground large language models, yet the methods that address it have multiplied across communities until they are seldom read as a single field.

By Sean Moran
llmsragbenchmarks
More like this →
arXiv AI
Oct 5

MRVQ: One Resident Index for Dimension- and Rate-Elastic Vector Search

MRVQ (Matryoshka Residual Vector Quantization) is a single post‑hoc residual quantizer designed for frozen embeddings that can be truncated to support any combination of embedding dimension and code rate. By dropping residual stages or embedding coordinates, MRVQ provides a single resident artifact that serves all evaluated (dimension, rate) pairs, achieving the lowest memory usage among tested designs. Although it uses 17.8–22.0× less memory than three separately trained QINCo2 indices and 1.89–2.02× less than a lean shared‑model steelman, it trades off a small drop in nDCG@10 on FiQA compared to per‑rate QINCo2, yet still outperforms PQ, OPQ, and AdANNS‑OPQ at matched code size.

By Sean Culatana, Shang-En Huang, Kang Li
ragefficiency
More like this →
arXiv AI
Oct 2

HHR: Hierarchical Hash Retrieval for Efficient LLM Generation

The paper introduces Hierarchical Hash Retrieval (HHR), a coarse‑to‑fine framework designed to improve hash‑based retrieval for large language models. HHR combines Geometry‑Aware Key Routing (GKR) to redistribute feature magnitudes and prune low‑logit keys, with Learned Hash Projection (LHP) to align Hamming distance with true query‑key relevance for fine‑grained retrieval. Experiments on diverse LLMs and benchmarks show that HHR outperforms existing methods, boosting LongBench scores by 1.10 points and achieving up to 3.30× decoding speedup at 128K context length for Llama‑3.1‑8B‑Instruct.

By Lianjun Liu, Tiantian Zheng, You Huang, Weiqi Yan, Mingte Qiu, Huazhong Liu, Xiaofeng Zhu, Yunshan Zhong
llmsefficiencybenchmarks
More like this →
arXiv AI
Jul 21

TurboVec: A Case Study in Cost-Efficient Private Retrieval for Enterprise RAG via Codebook-Oblivious Quantization

arXiv:2607. 16973v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) systems increasingly power enterprise LLM applications, yet the vector retrieval layer introduces two underexplored challenges: (1) trained codebook quantizers may expose corpus statistics during index construction, creating a leakage channel in multi-tenant deployments, and (2) post-hoc filtering for tenant isolation degrades recall on selective queries.

By Navnit Shukla, Kamal Pandey, Omsankar Tiwari
llmsragefficiencybenchmarkssafety
More like this →