arXiv Computation and Language October 16, 2026 By Shuo Lin, Jinzhe Li, Chenlu Guo, Yanhan Li, Yuan Wu, Yi Chang

PremiseSQL: Evaluating Premise Critique and Abstention in Text-to-SQL Systems

Read the original on arXiv Computation and Language →

PremiseSQL introduces a benchmark of 1,800 flawed natural language to SQL (NL2SQL) queries, grouped into four categories of premise failures, alongside their original valid counterparts. The study evaluates large language models’ ability to critique flawed premises and abstain from generating misleading SQL, using an evaluation framework across three prompting settings. Results show that current models tend to trust premises and rarely critique them, while stronger critique-oriented prompting improves detection but also raises false invalid judgments and reduces SQL generation for valid queries.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

4d ago

Reliable Text-to-SQL in Resource-Constrained Settings with Small LLMs via Iterative Retry Workflows

arXiv:2610.17305v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently improved Text-to-SQL generation, but their computational cost can hinder deployment in resource-constrained...

By Bilel Abderrahmane Benziane, Xavier Schepler

llms benchmarks data-engineering

Hugging Face Trending Papers

Aug 26

From Passive Response to Proactive Correction: Enhancing LLM Robustness Against Input Fact Perturbations

Large language models (LLMs) frequently produce confident yet factually incorrect responses when user inputs contain misleading premises, a phenomenon we attribute to fact perturbations in the input....

llms benchmarks safety

Hugging Face Trending Papers

Aug 14

Never the Number: Structural Abstention for AI Systems Whose Answers Are Consumed as Fact

Large language models have made natural language interfaces to databases (NLIDB) newly credible, but LLM text-to-SQL systems fail in a way that matters for deployment: a hallucinated column or a mis-a...

llms agents fine-tuning benchmarks data-engineering

arXiv Computation and Language

Aug 27

From Passive Response to Proactive Correction: Enhancing LLM Robustness Against Input Fact Perturbations

The paper introduces DEDUCE, a three‑stage framework that turns large language models into proactive error correctors by detecting input fact errors, devising correction strategies, and delivering reliable answers. It also presents MisFactQA, a dataset of factual errors, and new metrics for robustness evaluation. Experiments on TruthfulQA, FalseQA, and MisFactQA show significant gains in accuracy and error correction across Qwen, LLaMA, and Gemma models.

By Ping Wang, Xiangguo Sun, Bingbing Xu, Guocong Li, Xiaofeng Meng

llms benchmarks safety

arXiv AI

Aug 18

ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models

arXiv:2608. 15145v1 Announce Type: new Abstract: Large Language Models (LLMs) have been increasingly adopted in Text-to-SQL systems, yet SQL errors remain a major obstacle in real-world Text-to-SQL inference pipelines.

By Xinmei Huang, Jie Song, Peng Li, Fuxin Jiang, Jing Zhang, Tieying Zhang, Jianjun Chen, Chenming Liu, Tao Yang, Maoyin Liu, Wenda Li, Hong Chen, Cuiping Li

llms agents efficiency benchmarks data-engineering

arXiv AI

Sep 10

SQLMorph: Query Mutation and Fine-Grained Metrics for Text-to-SQL Evaluation

arXiv:2609.08950v1 Announce Type: cross Abstract: Text-to-SQL systems translate natural language queries into executable SQL, democratizing access to structured data. Despite recent advances driven b...

By Mohammadhossein Malekpour, Mohamed Riahi, Maxime Lamothe, Amine Mhedhbi

llms benchmarks data-engineering

One email a morning, machine-written

Related stories

Reliable Text-to-SQL in Resource-Constrained Settings with Small LLMs via Iterative Retry Workflows

From Passive Response to Proactive Correction: Enhancing LLM Robustness Against Input Fact Perturbations

Never the Number: Structural Abstention for AI Systems Whose Answers Are Consumed as Fact

From Passive Response to Proactive Correction: Enhancing LLM Robustness Against Input Fact Perturbations

ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models

SQLMorph: Query Mutation and Fine-Grained Metrics for Text-to-SQL Evaluation