Retrieval-augmented generation

Retrieval pipelines, vector search, chunking and reranking: how models are grounded in a corpus instead of their weights.

6,383 stories · RSS feed

arXiv Computer Vision
Aug 27

On the Separation of Human and AI-Generated Images in CLIP Embedding Space

The paper reports a new phenomenon in CLIP embeddings where human and AI‑generated paintings naturally separate along dominant principal directions without any supervised training. The authors investigate this separation by linking embedding directions back to image features using interpretable representations and gradient‑based inversion, finding that the separation is driven by distributed multiscale image structure rather than simple global or local statistics. They also show that small, imperceptible image perturbations can cause large displacements along these directions, highlighting a mismatch between CLIP’s visual evidence and human perception.

By Andrea Asperti
arXiv Computation and Language
Aug 27

Generative vs. Encoder Large Language Models for ASR Evaluation: A Comparative Study

The paper compares encoder‑based and generative decoder‑based large language models for evaluating automatic speech recognition (ASR). It examines BERTScore and SemDist across various LLMs, layers, and pooling strategies, finding that both metrics can strongly correlate with human judgments when properly configured. For generative LLMs, the study explores pairwise hypothesis selection via prompting and direct error classification, showing that while encoder‑based metrics remain competitive, generative models excel in hypothesis comparison and enhance interpretability of ASR evaluation.

By Thibault Ba\~neras-Roux, Shashi Kumar, Driss Khalil, Sergio Burdisso, Petr Motlicek, Shiran Liu, Mickael Rouvier, Jane Wottawa, Richard Dufour
arXiv Computation and Language
Aug 27

AWM: Answerable Working Memory for Long-Document VQA Agents

The paper introduces AWM, a framework that treats the terminal working memory of long‑document VQA agents as an answerable evidence artifact. It proposes a memory‑only answerability diagnostic and incorporates this signal into the GRPO reward, giving higher advantage to trajectories whose final memory can answer the question alone. Experiments on MMLongBench‑Doc and LongDocURL show that AWM‑GRPO boosts final‑answer accuracy by up to 11.9 points and reduces the rate of correct answers that cannot be supported by memory alone.

By Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu, Zhen Yang, Rui Lu, Yuxiao Dong, Jie Tang, Evgeny Kharlamov
arXiv Computation and Language
Aug 27

Corpus2Skill: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG

Corpus2Skill is a retrieval architecture that transforms an enterprise knowledge base into a hierarchical skill directory, enabling an LLM agent to navigate from high-level summaries to specific documents and backtrack when necessary. On an enterprise customer‑support benchmark, it outperforms single‑shot dense, hybrid, hierarchical‑retrieval, and agentic RAG baselines in answer quality and grounding, with a moderate cost tradeoff. An eleven‑dataset study shows that corpus navigation excels on single‑domain corpora with a recoverable topical taxonomy but is less effective on open‑domain factoid pools or homogeneous‑tabular corpora, providing a design guideline for knowledge‑grounded systems.

By Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh
arXiv Machine Learning
Aug 27

Are LLM-Enhanced GNNs Privacy-Safe?

The paper evaluates privacy risks in graph neural networks enhanced by large language models (LLMs). Using a five‑stage framework, the authors test six real‑world text‑attributed graph datasets with 42 model configurations and six privacy attack methods across link, label, and membership inference threats. Results show that LLM‑enhanced GNNs are more vulnerable than shallow baselines, with semantic enrichment amplifying exploitable signals, and that differential privacy can reduce risk but at a significant cost to utility.

By Longzhu He, Zelang Wen, Chaozhuo Li, Sen Su
arXiv Computation and Language
Aug 27

VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval

The paper introduces VDAR-Router, a routing framework for large language models that uses verbalized query difficulty analysis to guide model selection. It first generates an explicit difficulty profile for each query, retrieves historical examples with similar profiles, and then estimates model suitability to choose a model based on a reward function balancing performance and cost. Experiments on three datasets show that VDAR-Router consistently outperforms existing baselines in cost‑performance trade‑offs, and case studies confirm that explicit difficulty analysis improves example relevance and routing reliability.

By Yu-Chien Tang, Jun-Chen Hung, Wen-Chih Peng, An-Zi Yen
arXiv Machine Learning
Aug 27

Why Does Graph Learning Fail to Fully Benefit from a Text Teacher?

The paper examines a multimodal approach that combines a self‑supervised GNN encoder with an alternating optimization scheme involving a language‑model teacher. Despite the expectation that this joint strategy would enhance predictive performance, the authors find that the combined model fails to deliver significant gains. They identify six key factors—ranging from anchor strength trade‑offs to misaligned representation spaces—that explain why the integration of text knowledge does not fully benefit graph learning.

By Fumiaki Kimino (SOKENDAI), Ryoma Sato (SOKENDAI, National Institute of Informatics)
arXiv Computation and Language
Aug 27

E2Rank: Unifying Text Embedding and Listwise Reranking for Effective and Efficient Search

E2Rank (Efficient Embedding-based Ranking) is a unified framework that extends a single text embedding model to perform both retrieval and listwise reranking. By treating the listwise prompt—constructed from the query and its top‑K candidates—as a pseudo‑relevance feedback query, E2Rank reranks via cosine similarity against precomputed document embeddings, avoiding costly autoregressive decoding. The approach achieves state‑of‑the‑art results on BEIR, competitive performance on the reasoning‑intensive BRIGHT benchmark, lower latency than existing LLM‑based rerankers, and improved embedding performance on MTEB—all within a single model.

By Qi Liu, Yanzhao Zhang, Mingxin Li, Dingkun Long, Pengjun Xie, Jiaxin Mao
arXiv Machine Learning
Aug 27

Model-Agnostic Open-Set Air-to-Air Visual Object Detection for Reliable UAV Perception

The paper introduces a model‑agnostic open‑set detection framework for air‑to‑air visual object detection on UAVs, addressing the limitations of closed‑set detectors under domain shifts and flight data corruption. It estimates semantic uncertainty through entropy modeling in the embedding space and employs spectral normalization and temperature scaling to improve open‑set discrimination. Experiments on the AOT aerial benchmark and real‑world flight tests show up to a 10% relative AUROC improvement over standard YOLO detectors, with background rejection further enhancing robustness without sacrificing accuracy.

By Spyridon Loukovitis, Anastasios Arsenos, Vasileios Karampinis, Athanasios Voulodimos
arXiv Machine Learning
Aug 27

Improved Analysis for Hessian-free High-resolution Monte Carlo Sampling

The paper introduces Hessian-free high-resolution (HFHR) dynamics, an extension of underdamped Langevin dynamics that incorporates reversible position diffusion for sampling in machine learning. It provides an explicit quantitative contraction rate under a position Poincaré inequality, weighted Hessian and Laplacian bounds, and a compact Sobolev embedding, even when the potential is non‑convex. For the HFHR Monte Carlo algorithm, a path‑space Girsanov argument yields a non‑asymptotic convergence bound and an explicit iteration complexity in total variation distance, improving on previous HFHR results and demonstrating benefits of a positive diffusion parameter through numerical experiments.

By Wujun Lv, Xiaoyu Wang, Yingli Wang, Lingjiong Zhu
Towards Data Science
Aug 26

How Does a RAG Reranker Really Work?

The article "How Does a RAG Reranker Really Work?" explores the inner workings of Retrieval-Augmented Generation (RAG) rerankers, focusing on how data scientists explain the model’s operations behind the scenes. It discusses the impact of these insights on architecture decisions within enterprise document intelligence, specifically in the context of Enterprise Document Intelligence Vol.1 #2D. The piece highlights the importance of transparent model explanations for effective enterprise RAG implementation.

By Kezhan Shi
arXiv AI
Aug 26

Retrieval-augmented generation vs. deterministic tax computation in multi-agent financial advisory: A 2x2 factorial experiment

The study compares a custom capital gains calculation engine with a retrieval‑augmented generation (RAG) vector store of market advisory reports in a multi‑agent financial advisory system. A 2x2 factorial experiment showed that the tax‑optimization engine significantly reduced tax savings, while the RAG component had no significant effect. The RAG‑only condition yielded the highest tax savings, suggesting that pretrained language model knowledge may suffice for tax‑loss harvesting without specialized tooling.

By Aryan Brar, Justin Du, Avery Lor, Kylie Seto, Eric Taylor