← Back to all news
arXiv AI

NL2SQL with Iterative Self-Refinement

The paper presents a multi‑agent, persona‑driven NL2SQL framework that addresses common LLM shortcomings such as schema hallucination, syntax errors, and intent mismatch. It incorporates domain relevance detection, schema‑grounded knowledge construction, KPI‑aware SQL synthesis, syntax‑level self‑repair via sqlite feedback, and semantic consistency checking through SQL‑to‑NL reverse engineering. Evaluated on 60 queries, the system achieves 90% correctness using Mistral across seven coordinated LLM calls, demonstrating a significant improvement in reliability for high‑stakes environments.

Related stories

arXiv Machine Learning
Sep 22

Dial: A Knowledge-Grounded Dialect-Specific NL2SQL System

The paper introduces Dial, a knowledge‑grounded framework for generating SQL queries that respect the specific dialect of heterogeneous database systems. Dial comprises a Dialect‑Aware Logical Query Planning module, a hierarchical intent‑aware knowledge base (HINT‑KB) containing canonical syntax, function, and constraint repositories, and an execution‑driven debugging loop that separates syntactic recovery from logic auditing. The authors also present DS‑NL2SQL, a benchmark of 2,218 dialect‑specific test cases across six major database systems, and report that Dial improves translation accuracy by 10.25% and dialect feature coverage by 15.77% over existing baselines.

By Xiang Zhang, Hongming Xu, Le Zhou, Wei Zhou, Xuanhe Zhou, Guoliang Li, Yuyu Luo, Changdong Liu, Guorun Chen, Jiang Liao, Fan Wu
ragfine-tuningbenchmarksdata-engineering
More like this →
arXiv AI
Sep 4

Reflect-SQL: A Self-Reflection Based Framework for Text-to-SQL

Reflect‑SQL is a new framework for converting natural language into SQL queries. It tackles challenges such as large, obscure database schemas, poor table and column retrieval, and syntactically or logically flawed SQL by using a multi‑stage self‑reflection approach. The system iteratively refines queries and SQL through feedback loops driven by an LLM‑as‑a‑judge, achieving 72.03% execution accuracy on the BIRD benchmark, outperforming existing baselines.

By Anupreksha Jain, Manish Shrivastava
llmsragbenchmarksdata-engineering
More like this →
arXiv AI
Sep 7

A Cost-Aware Agentic Architecture for NL-to-SQL over Nested Enterprise Schemas, with a New Benchmark

The paper introduces the DevRev NL2SQL benchmark, featuring 900 execution‑verified queries that test natural‑language‑to‑SQL systems on nested, graph‑like enterprise schemas, and proposes the Semantic Depth Score (SDS) as a rubric for analytical reasoning depth. It also presents a cost‑aware, single‑generation agentic architecture that includes schema selection, metadata retrieval, and error‑repair components tailored to these complex schemas. On the DevRev benchmark, the system achieves 91.7% answer correctness, outperforming the next‑best baseline by 54.6 percentage points, and remains competitive on the Spider 2.0 Snowflake dataset.

By Yoga Sri Varshan Varadharajan, Ajay Yadav, Ritesh Goru, Prateek Chaudhury, Constantine Caramanis, Prateek Jain, Divyateja Pasupuleti, Sunil Kumar Pandey
agentsbenchmarksdata-engineering
More like this →
arXiv Computation and Language
Oct 16

PremiseSQL: Evaluating Premise Critique and Abstention in Text-to-SQL Systems

PremiseSQL introduces a benchmark of 1,800 flawed natural language to SQL (NL2SQL) queries, grouped into four categories of premise failures, alongside their original valid counterparts. The study evaluates large language models’ ability to critique flawed premises and abstain from generating misleading SQL, using an evaluation framework across three prompting settings. Results show that current models tend to trust premises and rarely critique them, while stronger critique-oriented prompting improves detection but also raises false invalid judgments and reduces SQL generation for valid queries.

By Shuo Lin, Jinzhe Li, Chenlu Guo, Yanhan Li, Yuan Wu, Yi Chang
llmsbenchmarksdata-engineering
More like this →