arXiv Machine Learning

Dial: A Knowledge-Grounded Dialect-Specific NL2SQL System

The paper introduces Dial, a knowledge‑grounded framework for generating SQL queries that respect the specific dialect of heterogeneous database systems. Dial comprises a Dialect‑Aware Logical Query Planning module, a hierarchical intent‑aware knowledge base (HINT‑KB) containing canonical syntax, function, and constraint repositories, and an execution‑driven debugging loop that separates syntactic recovery from logic auditing. The authors also present DS‑NL2SQL, a benchmark of 2,218 dialect‑specific test cases across six major database systems, and report that Dial improves translation accuracy by 10.25% and dialect feature coverage by 15.77% over existing baselines.

arXiv AI
Oct 19

NL2SQL with Iterative Self-Refinement

The paper presents a multi‑agent, persona‑driven NL2SQL framework that addresses common LLM shortcomings such as schema hallucination, syntax errors, and intent mismatch. It incorporates domain relevance detection, schema‑grounded knowledge construction, KPI‑aware SQL synthesis, syntax‑level self‑repair via sqlite feedback, and semantic consistency checking through SQL‑to‑NL reverse engineering. Evaluated on 60 queries, the system achieves 90% correctness using Mistral across seven coordinated LLM calls, demonstrating a significant improvement in reliability for high‑stakes environments.

By Anjali Vemula, Akshat Dasula
arXiv AI
Sep 10

ProcArena: A Multi-Scenario Benchmark for LLMs on Direct and Interactive PL/SQL Development from Natural Language

ProcArena is a new benchmark for evaluating large language models on natural‑language to PL/SQL translation tasks. It contains 3,998 executable tasks across 157 databases, covering nine development subscenarios in PostgreSQL and Oracle, and supports both direct generation and interactive multi‑turn scenarios. Experiments on seven models show that even the best performers achieve only about 62% accuracy in direct mode and 58% in interactive mode, highlighting the difficulty of realistic NL‑to‑PL/SQL development.

By Hang Zhang, Chaokun Wang, Yuzhi Pan, Ziyao Zhong, Shuo Cao, Yue Xue, Zeyu Huang, Xingwei Zhou, Fang Niu, Bofan Xie, Guanchen Ge, Leqi Zheng, Ziyang Liu, Xiannian Cao, Pengcheng Ge
arXiv Computation and Language
Oct 8

ARCS: Towards Precise Text-to-SQL via Structured Disambiguation

The paper introduces ARCS, a new benchmark for text-to-SQL that focuses on naturally occurring ambiguities in user queries over real-world databases. It proposes a structured disambiguation approach, where ambiguity is resolved through explicit, constrained interactions instead of free-form dialogue. Experiments show that even advanced models struggle, with the best open-source model achieving only 27% execution accuracy.

By Yihao Hu, Yanlin Feng, Naoki Otani, Nikita Bhutani
arXiv Machine Learning
Sep 11

Enabling Knowledge Graph Understanding at Scale with the EXplore Your Graphs ENgine (EXYGEN)

The paper introduces EXYGEN, a framework that enables conversational access to large knowledge graphs by combining VoID descriptions, ShEx schemas, retrieved triples, and example question‑query pairs in a retrieval‑augmented generation pipeline. On the SciQA benchmark, this approach achieves an exact‑match score of 0.419 without fine‑tuning any large language model, and shows that larger general‑purpose LLMs can outperform smaller code‑specialized ones when provided sufficient context. To scale metadata generation for very large KGs, the authors propose a predicate‑coverage‑aware parallel graph sampling strategy that preserves structural diversity, reduces runtime by over 80× on OpenCitations Meta and GESIS, and is the only tractable method for obtaining complete metadata on ORKG.

By Harshdeep Singh, Yurui Zhu, Giovanni Colavizza, Matteo Romanello