← Back to all news
arXiv AI By Jingyuan Wang, Richong Zhang, Zhijie Nie, Mingxin Li, Yanzhao Zhang

DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval

Read the original on arXiv AI →

The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Jun 15

Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA

arXiv:2604. 23336v3 Announce Type: replace-cross Abstract: Unlike traditional fact-based retrieval, rationale-based retrieval typically necessitates cross-encoding of query-document pairs using large language models, incurring substantial computational costs.

By Teng Chen, Sheng Xu, Feixiang Guo, Xiaoyu Wang, Qingqing Gu, Hongyan Li, Luo Ji
llmsragroboticsefficiencybenchmarks
More like this →
arXiv AI
Aug 25

Unleashing the Power of LLMs in Dense Retrieval with Query Likelihood Modeling

The paper introduces LLM-QL, a dense retrieval model that harnesses large language models (LLMs) by maximizing query likelihood (QL) as an auxiliary task. It incorporates an Attention Block to limit predictive token attention to document tokens before the ending token and a Document Corruption component that masks parts of the document during prediction. Experiments on MS MARCO and BEIR datasets show that LLM-QL outperforms other LLM-based retrievers, and detailed analyses confirm the effectiveness of its components.

By Hengran Zhang, Keping Bi, Jiafeng Guo, Xiaojie Sun, Shihao Liu, Daiting Shi, Dawei Yin, Xueqi Cheng
llms
More like this →
arXiv AI
Aug 11

TreeHop: Efficient Embedding-Level Query Rewriter

arXiv:2504. 20114v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) systems face significant challenges in multi-hop question answering (MHQA), where complex queries require synthesizing information across multiple document chunks.

By Zhonghao Li, Kunpeng Zhang, Jinghuai Ou, Shuliang Liu, Xuming Hu
llmsragnlp
More like this →