arXiv:2610.30705v1 Announce Type: new Abstract: Text embedding and text generation are two of the most useful capabilities of modern large language models, yet they are typically provided by separate...
The paper introduces DEPT, a method that trains a single decoder-only large language model to both expand queries and encode documents for retrieval. By preserving document embeddings close to their initial cached values while allowing gradients to flow through the generator, DEPT stabilizes retrieval targets and enables efficient index reuse and online hard‑negative mining. Experiments on the BEIR benchmark with Qwen3‑4B‑Instruct‑2507 and LLaMA‑3.2‑3B‑Instruct show that DEPT outperforms training‑free, independently trained, and staged unified baselines, with ablations confirming the benefits of preservation, whitening, end‑to‑end expansion training, and online negatives.
The paper introduces Retrieval-Augmented Decoding (RAD), a decoding-time method that improves the truthfulness of large language models without retraining. RAD uses a small reference set of up to ten annotated examples to build a grounding space of context embeddings and next-token logits, which it retrieves and aggregates during inference to shape the model’s output. Experiments on four open-ended generation benchmarks and four different LLMs show that RAD consistently outperforms strong baselines and generalizes well across tasks.
arXiv:2608.30627v1 Announce Type: new Abstract: As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token pred...
The paper introduces Lens, a training‑free framework that aligns multimodal representations with the semantic perspective required by downstream tasks. Lens uses a task‑specific readout phrase to anchor the perspective and then aggregates token states after the full input, ensuring the extracted representation reflects task‑conditioned evidence integration rather than generic salient content. The method achieves a Precision@1 of 63.9 across 36 MMEB datasets, outperforming the nearest training‑free baseline by 10.2 points.
MEA is a multi‑agent framework that automates the selection and configuration of explanation tools for machine learning models across tabular, text, and vision data. A Proposer agent chooses appropriate tools based on the question and modality, while an Actor agent is trained end‑to‑end for faithfulness, producing natural‑language explanations. Experiments show that MEA outperforms existing post‑hoc explainers and closed‑source baselines, achieving significant faithfulness gains on six datasets.
arXiv:2511. 20639v3 Announce Type: replace-cross Abstract: Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence.
arXiv:2609.23371v1 Announce Type: cross Abstract: Long-context language models interface with external knowledge through raw natural language. In retrieval-augmented systems, this creates a persisten...
LiSeCo is a lightweight, gradient‑free method that controls language generation by directly intervening on the hidden activations of a token in embedding space. It uses control‑theoretic techniques to steer the generation trajectory away from undesired semantic regions and into a predefined allowed region, ensuring fine‑grained attribute control. The approach is computationally efficient, minimally impacts generation time, and is shown to be effective on tasks such as toxicity, sentiment, and bilingual language steering while preserving text quality.
arXiv:2610.01936v2 Announce Type: replace Abstract: Large Language Models (LLMs) have demonstrated remarkable fluency and versatility across natural language tasks but remain fundamentally limited by...
arXiv:2609.22162v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves the factuality of large language models (LLMs) and vision-language models (VLMs) by grounding generatio...
arXiv:2610.07731v1 Announce Type: cross Abstract: Dense retrieval models are typically trained with contrastive objectives that learn effective representations but do not directly optimize retrieval...