arXiv AI

Semantically Similar, Logically Distinct: Diagnosing the Semantic-Answerability Gap in Table RAG

arXiv:2607. 17742v1 Announce Type: new Abstract: Tables are a critical knowledge source in retrieval-augmented generation (RAG), but a retrieved table may lack sufficient evidence to answer a query, a property we call answerability.

arXiv AI
2d ago

BudgetSchemaBench: A Budget-Swept Diagnostic for Schema Context in Text-to-SQL

BudgetSchemaBench is a diagnostic tool for evaluating how different schema‑context budgets affect text‑to‑SQL systems. It automatically derives relevance labels from gold SQL, tests four budgets across 80 databases, and compares three schema representations while keeping table rankings fixed. The study shows that increasing the budget improves execution accuracy, especially for lexical retrieval, and that dense retrieval already captures most needed tables at low budgets.

By Chen Shen
arXiv AI
Aug 24

Clarify-Then-Search: A Clarification Benchmark for Deep Search with End-to-End Nugget Restoration

Clarify-Then-Search is a benchmark that tests whether large language models can ask clarification questions to improve the usefulness of deep search results. It uses 518 real-world query pairs from Baidu, where each intent query is paired with an underspecified version. The evaluation involves a clarifier asking up to three questions, a user answerer providing only explicit information, and a rewriter generating a new query that is then searched; performance is measured by a weighted nugget-recall score.

By Deqiang Huang, Jingbo Zhou, Xinjiang Lu, Tong Xu, Hua Wu, Enhong Chen
arXiv AI
Aug 26

PARTAB: Partition-Aware Reasoning with Structured Evidence for Scalable Table Understanding

PARTAB is a framework that improves large language model reasoning on tables by constructing a structured evidence interface. It represents query‑relevant evidence as semantically coherent, row‑linked table regions and performs hierarchical selection over column groups and row‑level partitions before composing the evidence for answer generation. Evaluations on multiple table reasoning benchmarks show that PARTAB consistently outperforms full‑table prompting and recent methods, achieving strong performance on WikiTableQuestions and TabFact while remaining competitive on numerical reasoning tasks.

By Md Mahadi Hasan Nahid, Davood Rafiei
arXiv AI
Jul 20

Ruling Out to Rule In: Contrastive Hypothesis Retrieval for Medical Question Answering

arXiv:2604. 04593v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds large language models in external medical knowledge, yet standard retrievers frequently surface hard negatives that are semantically close to the query but describe clinically distinct conditions.

By Byeolhee Kim, Min-Kyung Kim, Young-Hak Kim, Tae-Joon Jeon
arXiv AI
Sep 16

Diagnosing the Fact-Grounding Gap in Multi-Hop Question Answering

The paper investigates multi‑hop question answering systems and identifies two distinct failure modes: retrieval failures, where the necessary passage is not retrieved, and extraction failures, where the passage is retrieved but the required fact cannot be extracted—a phenomenon termed the fact‑grounding gap. Across three standard benchmarks, extraction failures account for nearly half of all per‑hop deficiencies and are invisible to standard retrieval metrics, remaining unresolved by retrieval‑only interventions. The study shows that these two bottlenecks require different solutions, a distinction currently missing from evaluation practices.

By Kevin Mo, Nathan Mo, Richard Zhu
arXiv AI
Aug 26

ESQ-Bench: A Multi-Tier Enterprise Oracle Benchmark for Evaluating NL2SQL Dialect Generalization and Silent Semantic Divergence

ESQ‑Bench is an Oracle‑first NL2SQL benchmark that introduces systematic complexity tiers and silent‑divergence evaluation across three enterprise schema levels. It provides six populated schemas (465 tables, 164,682 rows) on Oracle, PostgreSQL, MySQL, and SQL Server, along with 550 gold‑validated question‑query pairs and a four‑metric evaluation harness. The benchmark reveals that state‑of‑the‑art models such as GPT‑4o and Claude Sonnet 4.6 experience significant drops in execution accuracy and high silent‑divergence rates as schema complexity increases, highlighting a gap between closed‑API models and open‑weight baselines on enterprise Oracle schemas.

By Sanjay Mishra, Divya Chukkapalli, Ganesh R. Naik