The paper introduces SYNTRA, a self‑improving framework that transforms a small set of retrieval examples into reusable synthesis instructions for generating training data. By inferring implicit information needs and relevance criteria from an unlabelled corpus and refining instructions through feedback, SYNTRA produces effective training data across 24 datasets and 15 domains, improving retrieval performance even for strong embedding models. Experiments show that SYNTRA’s self‑improvement yields better results than human calibration on four tasks and scales effectively, with continued gains on MS MARCO across a 32‑fold range of synthetic data sizes.
By Zucan Lyu, Rui Tang, Jianlyu Chen, Peisheng Jin, Defu Lian
arXiv:2604. 26170v2 Announce Type: replace Abstract: Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.
By Ting-Wei Li, Sirui Chen, Jiaru Zou, Yingbing Huang, Tianxin Wei, Jingrui He, Hanghang Tong
Prompted embedding models have recently received increasing attention, particularly for retrieval, where detailed retrieval instructions are provided as part of the retrieval prompt. Several new datas...
arXiv:2610.10508v1 Announce Type: new
Abstract: Prompted embedding models have recently received increasing attention, particularly for retrieval, where detailed retrieval instructions are provided a...
By Amanda Myntti, Jenna Kanerva, Veronika Laippala, Filip Ginter
The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.
By Jingyuan Wang, Richong Zhang, Zhijie Nie, Mingxin Li, Yanzhao Zhang
The paper introduces INTRA, an attention-based encoder-decoder framework that retrieves directly from its own internal representations instead of using an external retriever. By having decoder attention query pre-encoded evidence chunks, INTRA unifies retrieval and generation, eliminating the typical mismatch seen in retrieval-augmented generation pipelines. Experiments on question-answering benchmarks show that INTRA outperforms strong engineered retrieval pipelines in both evidence recall and overall answer quality.
By Elad Hoffer, Yochai Blau, Edan Kinderman, Ron Banner, Daniel Soudry, Boris Ginsburg
The paper introduces Efficient Retrieval Adapter (ERA), a query‑side adapter framework that enables dense retrieval systems to adapt to asymmetric query–document scenarios without re‑indexing. ERA first aligns the embedding spaces of a powerful query embedder and a lightweight document embedder using unlabeled documents, then fine‑tunes the aligned query representation with a small set of labeled query‑document pairs. In experiments on 126 MAIR retrieval tasks across six domains, ERA boosts average nDCG@10 by up to 8.2 points in symmetric settings and over 12 points in asymmetric settings while requiring far fewer labels than fully supervised adapter training.
By Seiji Maekawa, Moin Aminnaseri, Pouya Pezeshkpour, Estevam Hruschka
Upgrading embedding models typically requires expensive database re-indexing, as new query embeddings are incompatible with existing database embeddings. While Backward Compatible Training (BCT) mitig...
arXiv:2503.06868v2 Announce Type: replace-cross
Abstract: Existing long-text generation methods produce lengthy outputs from short inputs, leaving long-input-to-long-output generation underexplored....
By Junhao Zhang, Richong Zhang, Fanshuang Kong, Ziyang Miao, Yanhan Ye, Yaowei Zheng
arXiv:2610.07572v1 Announce Type: cross
Abstract: In-context learning (ICL) adapts frozen large multimodal models (LMMs) to new tasks from a few demonstrations (demos), but re-encodes them at every q...
By Xi Ding, Naichen Shi, Jiawei Zhang
arXiv:2606. 24841v1 Announce Type: new Abstract: Prompt-based learning has emerged as a dominant paradigm in natural language processing.
By Ahmad Pouramini, Hesham Faili
NE‑R1 is a framework that improves Named Entity Recognition by using a retrieval‑on‑demand mechanism and a two‑stage training process that includes instruction tuning and reinforcement learning with chain‑of‑thought. It balances the use of internal model parameters and external knowledge through a multi‑dimensional reward that considers accuracy and retrieval benefit. The approach achieves state‑of‑the‑art results, improving in‑domain F1 by 2.52% and zero‑shot cross‑domain F1 by 1.18%.
By Meixuan Chen, Hehan Li, Ruizhi Zhao, Xin Lu, peizhi xu, Liwei Qian, LI Meifang, shuanglong li, Hanmeng Liu, Xin Pei, Yanbiao Ma