arXiv Computation and Language

Select, Don't Train: The Benefits of Modular Entity Disambiguation with LLM-Based Selection

The paper investigates modular entity disambiguation by separating candidate retrieval from entity selection. It compares sparse retrieval (BM25), Web KB search, and a dense retriever, all paired with large language model selectors. Results show that a training‑free BM25 retriever combined with an LLM selector achieves state‑of‑the‑art performance on the ZELDA benchmark, and the modular approach enables abstention when retrieval fails.

arXiv AI
Aug 19

DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval

The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.

By Jingyuan Wang, Richong Zhang, Zhijie Nie, Mingxin Li, Yanzhao Zhang
arXiv AI
Sep 4

STAIR (STructure Aware Information Retriever): A novel dataset and LLM based retriever for document structure augmentation

The paper introduces STAIR, a retrieval system that uses a document’s Table of Contents to guide large language models in accessing global structure, thereby reducing hallucinations in Retrieval Augmented Generation. Experiments with a fine‑tuned Differentiable Search Index show that ToC‑based retrieval yields a low hallucination rate (<0.05%) and improves Recall@1 to 82.6% on the newly released SearchTome benchmark, outperforming baselines like BM25, DPR, and Mistral. The authors also release SearchTome, a diverse dataset of 18 books across six domains, to encourage further research in ToC‑based retrieval.

By Vineet Kumar, Meghanadh Pulivarthi, vishwajeet kumar, Jaydeep Sen, Riyaz Ahmad Bhat, Sachindra Joshi
arXiv Machine Learning
Aug 27

A Storage-Retrieval Gap in Parametric Knowledge Graph Memory

The paper investigates a parametric approach to knowledge graph memory by compiling each entity into a LoRA adapter, enabling zero‑cost query-time retrieval via weight injection. On the MetaQA dataset, these adapters encode context‑free factual knowledge, improving exact‑match scores by up to +0.243 over a base model and achieving an oracle gap of +0.283. However, the stored knowledge is not recoverable through similarity or embedding‑based methods, indicating that knowledge is stored locally and does not transfer across semantically neighboring entities.

By Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Volker Tresp
arXiv Computation and Language
Sep 23

BELXTR: Biomedical Entity Linking via Contextualized Token Retrieval

BELXTR is a new biomedical entity linking model that uses a multi‑vector (late interaction) architecture to preserve token‑level matching information, unlike traditional embedding‑based approaches that compress mentions into a single vector. By extending the XTR model with a task‑specific training objective and active query expansion, BELXTR achieves state‑of‑the‑art performance on half of ten evaluated corpora, with an average 5‑percentage‑point gain in recall@1. The model shows especially strong results on cross‑species gene disambiguation, outperforming an LLM‑powered retrieve‑and‑rerank pipeline and approaching a specialized rule‑based system.

By Samuele Garda, Ulf Leser
arXiv Machine Learning
Jun 15

Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA

arXiv:2604. 23336v3 Announce Type: replace-cross Abstract: Unlike traditional fact-based retrieval, rationale-based retrieval typically necessitates cross-encoding of query-document pairs using large language models, incurring substantial computational costs.

By Teng Chen, Sheng Xu, Feixiang Guo, Xiaoyu Wang, Qingqing Gu, Hongyan Li, Luo Ji