The paper argues that retrieval‑augmented question‑answering systems should perform semantic compilation at ingest time rather than re‑deriving meaning at query time. By building a maintained structure—incrementally updated embeddings and validated atomic claims—read operations become far cheaper, with experimental results showing higher accuracy and lower token usage compared to traditional chunk‑based retrieval. The authors present two proofs: cheaper incremental updates and superior performance on broadcast‑interview transcripts, suggesting a new systems agenda for compilation and read planning.
By Kyle Wild, Yusuke Takahashi, Asako Uraki
arXiv:2609.36550v1 Announce Type: new
Abstract: Retrieval-augmented generation is widely used in professional writing, yet whether retrieval grounds revision or merely injects templates is rarely tes...
By Josepha Michiko Leo, Hyun-seok Min, Yehoon Jang, Irvan Zidny, Jin-Woo Chung, Sungchul Choi
CARGO is a framework for evaluating agentic AI systems in production that addresses the problem of reference-instance divergence (RID), where reference-based judges penalize correct answers that involve different entity identifiers. It treats retrieved references as procedural exemplars, grounds judgments in the live instance’s context, assigns a three-way status to claims, and gates evaluation by retrieval confidence. Using the CARGO-Bench diagnostic suite, CARGO eliminates false penalties and improves discrimination while revealing a limitation in detecting procedural corruptions.
By Mukul Chhabra, Shail Patel, Luigi Medrano
arXiv:2606. 23695v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) grounds Large Language Models in external knowledge, yet current evaluations rely on discrete heuristics that suffer from ''epistemic blindness'' - failing to distinguish genuine contextual information extraction from parametric memory recall.
By Barak Or
arXiv:2609.38222v1 Announce Type: new
Abstract: Retrieval-augmented generation (RAG) can ground large language models in external evidence, but retrieved context does not guarantee that generated cla...
By Muhammad Aimal Rehman, Chi-Kuang Yeh
arXiv:2607. 02116v1 Announce Type: new Abstract: Autonomous AI agents increasingly depend on external knowledge stores, yet most retrieval pipelines provide relevance without durable guarantees of provenance, version identity, integrity, traceability, or point-in-time reconstruction.
By Misha Sulpovar (PromptOwl, LLC), Benn R. Konsynski (Goizueta Business School, Emory University), Qaish Kanchwala (IBM Research), Gabe Goodhart (IBM Research)
The paper evaluates how modern large language models use internal web search to answer factual questions. Using 783 static queries and 288 dynamic queries, the authors find that enabling retrieval improves accuracy on static questions but hurts confidence calibration. On dynamic queries, models often retrieve but still achieve less than 70% accuracy, mainly due to poor query formulation and source selection, indicating that internal web search works better as a quick verification tool than a full information‑retrieval system.
By Sahil Kale
BudgetSchemaBench is a diagnostic tool for evaluating how different schema‑context budgets affect text‑to‑SQL systems. It automatically derives relevance labels from gold SQL, tests four budgets across 80 databases, and compares three schema representations while keeping table rankings fixed. The study shows that increasing the budget improves execution accuracy, especially for lexical retrieval, and that dense retrieval already captures most needed tables at low budgets.
By Chen Shen
WildSEEK is a new dataset of 3,000 real user information‑seeking queries, manually annotated for risk‑sensitive domains and whether the query is factoid or analytical. The accompanying evaluation framework tests LLM responses against four failure criteria—sycophantic behavior, overreliance, a default US‑centric perspective, and poor handling of vulnerable populations—finding higher failure rates for analytical queries. The authors also train classifiers on WildSEEK to analyze over 1.8 million realistic queries, revealing that more than a third are high‑risk and often analytical.
By Tanise Ceron, Joachim Baumann, Elisa Bassignana, Berat Cabuk, Dirk Hovy, Debora Nozza
arXiv:2610.27580v1 Announce Type: cross
Abstract: Factual recall is a lookup: some representation inside the network serves as the key, and what that key is made of decides which queries can retrieve...
By Yuchen Guo, Zhuoyuan Mao, Yiwen Guo
arXiv:2609.22259v1 Announce Type: new
Abstract: Context layers, curated documentation that an analytics agent fetches at query time, produce large accuracy gains on text-to-SQL benchmarks. A with/wit...
By Qing Ye
arXiv:2604. 19047v2 Announce Type: replace-cross Abstract: Existing QA benchmarks typically assume distinct documents with minimal overlap, yet real-world retrieval-augmented generation (RAG) systems operate on corpora such as financial reports, legal codes, and patents, where information is highly redundant and documents exhibit strong inter-document similarity.
By Hanjun Cho, Jay-Yoon Lee