ESQ‑Bench is an Oracle‑first NL2SQL benchmark that introduces systematic complexity tiers and silent‑divergence evaluation across three enterprise schema levels. It provides six populated schemas (465 tables, 164,682 rows) on Oracle, PostgreSQL, MySQL, and SQL Server, along with 550 gold‑validated question‑query pairs and a four‑metric evaluation harness. The benchmark reveals that state‑of‑the‑art models such as GPT‑4o and Claude Sonnet 4.6 experience significant drops in execution accuracy and high silent‑divergence rates as schema complexity increases, highlighting a gap between closed‑API models and open‑weight baselines on enterprise Oracle schemas.
By Sanjay Mishra, Divya Chukkapalli, Ganesh R. Naik
arXiv:2607. 22572v1 Announce Type: new Abstract: Large language models can generate fluent SQL from natural language, but on real enterprise Oracle databases they frequently fail at execution time: columns and aliases are hallucinated and dialect-specific syntax is missed, leading to ORA-00904 invalid-identifier errors.
By Sanjay Mishra, Divya Chukkapalli, Ganesh R. Naik
arXiv:2608. 16663v1 Announce Type: cross Abstract: Direct text-to-SQL asks a language model to do two jobs: interpret the business question and construct the complete relational query.
By Yi Ai
arXiv:2607. 17269v1 Announce Type: new Abstract: Large language models encode world models implicitly in neural weights, which exposes four structural risks in high-precision domains such as medicine and finance: hallucination, frozen knowledge, poor explainability, and poor modifiability.
By Zhanbo Li, Shifeng Wu, Xiangjin Meng, Wenjie Cai
arXiv:2608. 19475v1 Announce Type: cross Abstract: This work introduces a formal semantic-block model for specifications and an execution-judged benchmark for evaluating specification quality independently of model capability.
By Oleg Grynets, Dmytro Kostetskyi, Vasyl Lyashkevych
arXiv:2606. 28387v1 Announce Type: cross Abstract: Enterprise text-to-SQL systems often fail before SQL is generated: the model receives the wrong schema context.
By Adarsh Agrawal, Shashank Indukuri