arXiv AI

Evaluating LLMs on Conversational Text-to-SQL under Chain Ambiguity and Intent Drift

arXiv AI

Sep 10

SQLMorph: Query Mutation and Fine-Grained Metrics for Text-to-SQL Evaluation

arXiv:2609.08950v1 Announce Type: cross Abstract: Text-to-SQL systems translate natural language queries into executable SQL, democratizing access to structured data. Despite recent advances driven b...

By Mohammadhossein Malekpour, Mohamed Riahi, Maxime Lamothe, Amine Mhedhbi

llms benchmarks data-engineering

arXiv AI

Aug 18

ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models

arXiv:2608. 15145v1 Announce Type: new Abstract: Large Language Models (LLMs) have been increasingly adopted in Text-to-SQL systems, yet SQL errors remain a major obstacle in real-world Text-to-SQL inference pipelines.

By Xinmei Huang, Jie Song, Peng Li, Fuxin Jiang, Jing Zhang, Tieying Zhang, Jianjun Chen, Chenming Liu, Tao Yang, Maoyin Liu, Wenda Li, Hong Chen, Cuiping Li

llms agents efficiency benchmarks data-engineering

arXiv AI

Jun 9

UniQL: Towards Dialect-Universal Benchmarking for Text-to-SQL

arXiv:2606. 08018v1 Announce Type: new Abstract: Existing text-to-SQL benchmarks are largely centered on SQLite, making it difficult to evaluate whether models can generalize across heterogeneous SQL dialects.

By Jianling Gao, Chongyang Tao, Jiayuan Bai, Liu Yang, Xuanguang Pan, Jinrui Liu, Shihao Xing, Xiaohan Xu, Jie Liang, Shuai Ma

llms nlp benchmarks data-engineering

arXiv Machine Learning

Sep 22

Dial: A Knowledge-Grounded Dialect-Specific NL2SQL System

The paper introduces Dial, a knowledge‑grounded framework for generating SQL queries that respect the specific dialect of heterogeneous database systems. Dial comprises a Dialect‑Aware Logical Query Planning module, a hierarchical intent‑aware knowledge base (HINT‑KB) containing canonical syntax, function, and constraint repositories, and an execution‑driven debugging loop that separates syntactic recovery from logic auditing. The authors also present DS‑NL2SQL, a benchmark of 2,218 dialect‑specific test cases across six major database systems, and report that Dial improves translation accuracy by 10.25% and dialect feature coverage by 15.77% over existing baselines.

By Xiang Zhang, Hongming Xu, Le Zhou, Wei Zhou, Xuanhe Zhou, Guoliang Li, Yuyu Luo, Changdong Liu, Guorun Chen, Jiang Liao, Fan Wu

rag fine-tuning benchmarks data-engineering

arXiv AI

Sep 21

The Stochastic Shift: A New Evaluation Paradigm for Text-to-SQL with AI Operators

arXiv:2609.21133v1 Announce Type: cross Abstract: SQL has been augmented with AI operators, enabling modern data analytics platforms to derive insights from both structured and unstructured data. We...

By Tarfah Alrashed, Fatma Ozcan, Per Jacobsson, Tal Neiman, Xianshun Chen

llms benchmarks data-engineering

arXiv Computation and Language

Aug 25

TraceSQL: Traceable Answerability Estimation for Reference-Free Text-to-SQL Verification

arXiv:2608.17795v2 Announce Type: replace Abstract: Text-to-SQL systems are commonly evaluated using ground-truth SQL queries or reference execution results, but such supervision is unavailable at in...

By Neelesh Kumar Shukla, Debasmita Panda, Srutanik Bhaduri, Aditya Banerjee, Vasu Rangarajan, Viji Krishnamurthy

llms agents safety data-engineering

arXiv AI

3d ago

Beyond Oracle Communication: Benchmarking Interactive Intent Alignment Under Miscommunication and Evolving User Intent

arXiv:2609.38604v1 Announce Type: cross Abstract: Modern LLM agents increasingly tackle complex tasks through interactive, long-horizon exchanges with users, while existing benchmarks generally assum...

By Zheyuan Zhang, Mengyuan Chao, Ke Xiao, Ziyi Chen, Daoan Zhang, Yan Zhang, Yanfang Ye, Wei Xu

llms agents benchmarks safety

arXiv AI

Jul 17

QDA-SQL: Questions Enhanced Dialogue Augmentation for Multi-Turn Text-to-SQL

arXiv:2406. 10593v3 Announce Type: replace Abstract: Fine-tuning large language models (LLMs) for specific domain tasks has achieved great success in Text-to-SQL tasks.

By Yinggang Sun, Ziming Guo, Haining Yu, Chuanyi Liu, Xiang Li, Bingxuan Wang, Xiangzhan Yu, Tiancheng Zhao

llms fine-tuning data-engineering

arXiv AI

Jun 16

Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning

arXiv:2606. 15598v1 Announce Type: new Abstract: Text-to-SQL aims to translate natural language questions into executable SQL queries over structured databases, enabling non-expert users to access data intuitively.

By Feng Lyu, Jinfeng Cen, Sijing Duan, Hao Wu, Shucheng Li, Weixu Zhang, Haolun Wu

llms fine-tuning benchmarks data-engineering

arXiv AI

Sep 1

BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations

arXiv:2608.29345v1 Announce Type: new Abstract: While recent Large Language Model (LLM)-based text-to-SQL systems achieve impressive performance on standard benchmarks, they struggle when user querie...

By Yunfan Zhou, Qiming Shi, Yizhou Yang, Di Weng, Yingcai Wu

llms benchmarks data-engineering

arXiv Machine Learning

Jul 24

LLMs Get Lost in Evolving User Intent

arXiv:2607. 20734v1 Announce Type: new Abstract: As LLMs become more capable, they are increasingly deployed as collaborative agents, taking on user-delegated tasks through iterative interaction.

By Jihoon Tack, Philippe Laban, Jennifer Neville

llms agents benchmarks

arXiv AI

Jun 11

TAHOE: Text-to-SQL with Automated Hint Optimization from Experience

arXiv:2606. 12387v1 Announce Type: cross Abstract: Large Language Models (LLMs) have democratized database access through Text-to-SQL, but moving from prototypes to production remains difficult.

By Zhiyi Chen, Jie Song, Peng Li

llms agents fine-tuning data-engineering

Related stories