arXiv AI
Beyond the Harness: End-to-End Optimization of Context Artifacts for Enterprise Text-to-SQL
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 26
ESQ-Bench: A Multi-Tier Enterprise Oracle Benchmark for Evaluating NL2SQL Dialect Generalization and Silent Semantic Divergence
ESQ‑Bench is an Oracle‑first NL2SQL benchmark that introduces systematic complexity tiers and silent‑divergence evaluation across three enterprise schema levels. It provides six populated schemas (465 tables, 164,682 rows) on Oracle, PostgreSQL, MySQL, and SQL Server, along with 550 gold‑validated question‑query pairs and a four‑metric evaluation harness. The benchmark reveals that state‑of‑the‑art models such as GPT‑4o and Claude Sonnet 4.6 experience significant drops in execution accuracy and high silent‑divergence rates as schema complexity increases, highlighting a gap between closed‑API models and open‑weight baselines on enterprise Oracle schemas.
arXiv AI
Aug 24
RAG Deserves an Index: Why Ingest-Time Compilation Beats Query-Time Interpretation
The paper argues that retrieval‑augmented question‑answering systems should perform semantic compilation at ingest time rather than re‑deriving meaning at query time. By building a maintained structure—incrementally updated embeddings and validated atomic claims—read operations become far cheaper, with experimental results showing higher accuracy and lower token usage compared to traditional chunk‑based retrieval. The authors present two proofs: cheaper incremental updates and superior performance on broadcast‑interview transcripts, suggesting a new systems agenda for compilation and read planning.
arXiv AI
Oct 2
BudgetSchemaBench: A Budget-Swept Diagnostic for Schema Context in Text-to-SQL
BudgetSchemaBench is a diagnostic tool for evaluating how different schema‑context budgets affect text‑to‑SQL systems. It automatically derives relevance labels from gold SQL, tests four budgets across 80 databases, and compares three schema representations while keeping table rankings fixed. The study shows that increasing the budget improves execution accuracy, especially for lexical retrieval, and that dense retrieval already captures most needed tables at low budgets.
arXiv AI
Jun 30
Schema-First Retrieval: Embedding Catalogs for Natural Language Analytics
arXiv:2606. 28387v1 Announce Type: cross Abstract: Enterprise text-to-SQL systems often fail before SQL is generated: the model receives the wrong schema context.
arXiv AI
Jun 30
Database Context Compression for Text-to-SQL on Real-World Large Databases
arXiv:2606. 28601v1 Announce Type: cross Abstract: Recent progress in Text-to-SQL has been driven by stronger language models and prompting strategies, yet performance on real enterprise benchmarks such as Spider 2.
arXiv AI
Oct 12
Semantic Views: Certified Reuse of LLM Labels Across Semantic Queries
arXiv:2610. 12768v1 Announce Type: cross Abstract: Semantic query engines evaluate natural-language filters with a large language model, and every evaluation is a costly oracle call.