The paper introduces Retrieval-Augmented Decoding (RAD), a decoding-time method that improves the truthfulness of large language models without retraining. RAD uses a small reference set of up to ten annotated examples to build a grounding space of context embeddings and next-token logits, which it retrieves and aggregates during inference to shape the model’s output. Experiments on four open-ended generation benchmarks and four different LLMs show that RAD consistently outperforms strong baselines and generalizes well across tasks.
arXiv:2604.17866v4 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) has become a standard approach for enhancing large language models (LLMs) with external knowledge, mitig...
arXiv:2608. 16515v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves large language models by grounding generation in external evidence, but it also introduces a source trust problem: retrieved context may be useful, irrelevant, or even misleading.
The paper introduces INTRA, an attention-based encoder-decoder framework that retrieves directly from its own internal representations instead of using an external retriever. By having decoder attention query pre-encoded evidence chunks, INTRA unifies retrieval and generation, eliminating the typical mismatch seen in retrieval-augmented generation pipelines. Experiments on question-answering benchmarks show that INTRA outperforms strong engineered retrieval pipelines in both evidence recall and overall answer quality.
arXiv:2609.22162v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves the factuality of large language models (LLMs) and vision-language models (VLMs) by grounding generatio...
arXiv:2610.01936v2 Announce Type: replace Abstract: Large Language Models (LLMs) have demonstrated remarkable fluency and versatility across natural language tasks but remain fundamentally limited by...
arXiv:2606. 00432v1 Announce Type: new Abstract: As retrieval-augmented generation (RAG) systems scale, it becomes increasingly challenging to ensure faithful grounding in external evidence.
arXiv:2505. 23277v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) often suffers from long and noisy retrieved contexts.
ULME introduces a post‑training framework that unifies text embedding and generation within a single causal decoder‑only LLM by inserting learnable slots for semantic and reasoning embeddings. It trains three objectives—contrastive, language‑modeling, and stop‑gradient predictive—so that generation actively shapes the embedding space. Across Qwen2.5 and Qwen3 models (0.5B–8B), ULME improves MTEB scores by 3.3–4.1 points, tightens BRIGHT and AdvBench‑IR, and matches generation‑only fine‑tunes on standard accuracy benchmarks.
arXiv:2603.09222v2 Announce Type: replace Abstract: Efficient context compression is critical for retrieval-augmented question answering in resource-constrained settings, where long retrieved context...
The paper proposes a new method for training large language models to handle long-context reasoning by combining Group Relative Policy Optimization (GRPO) with on‑policy distillation (OPD). It introduces a synthetic multilingual dataset called LongBlocks that tests multi‑hop reasoning, contextual grounding, and long‑form generation. Experiments show that the combined approach outperforms either GRPO or OPD alone while maintaining short‑context performance.
arXiv:2609.07093v2 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) enables large language models (LLMs) to answer questions by accessing external knowledge and has been widely a...
arXiv:2608.30426v1 Announce Type: new Abstract: Current dialogue systems struggle with dynamic information retrieval, often leading to hallucinations and lower response accuracy. We address this by a...