arXiv Computation and Language

Can LLMs Normalize Databases? A Benchmark and Multi-Agent Framework for Schema Normalization

The paper introduces a Database Normalization Benchmark (DNBENCH) with 3,275 samples to evaluate how well Large Language Models (LLMs) can perform database normalization from 1NF to BCNF, assessing semantic equivalence, structural accuracy, and logical validity. It identifies common failures in dependency inference, schema decomposition, and inter-table constraint reconstruction across various complexity levels. The authors also propose a Multi-Agent Reasoning for Schemas (MARS) framework that separates evidence extraction, violation diagnosis, and decomposition planning from schema generation, achieving an 82.0% improvement in DNB-SCORE over a single-prompt baseline.

arXiv AI
Sep 7

A Cost-Aware Agentic Architecture for NL-to-SQL over Nested Enterprise Schemas, with a New Benchmark

The paper introduces the DevRev NL2SQL benchmark, featuring 900 execution‑verified queries that test natural‑language‑to‑SQL systems on nested, graph‑like enterprise schemas, and proposes the Semantic Depth Score (SDS) as a rubric for analytical reasoning depth. It also presents a cost‑aware, single‑generation agentic architecture that includes schema selection, metadata retrieval, and error‑repair components tailored to these complex schemas. On the DevRev benchmark, the system achieves 91.7% answer correctness, outperforming the next‑best baseline by 54.6 percentage points, and remains competitive on the Spider 2.0 Snowflake dataset.

By Yoga Sri Varshan Varadharajan, Ajay Yadav, Ritesh Goru, Prateek Chaudhury, Constantine Caramanis, Prateek Jain, Divyateja Pasupuleti, Sunil Kumar Pandey
Hugging Face Trending Papers
Sep 4

A Cost-Aware Agentic Architecture for NL-to-SQL over Nested Enterprise Schemas, with a New Benchmark

The paper introduces the DevRev NL2SQL benchmark, comprising 900 execution‑verified queries that feature nested‑type and link‑graph structures, along with the Semantic Depth Score (SDS) to assess analytical reasoning depth. It also presents a cost‑aware single‑generation agentic architecture that includes schema selection, metadata retrieval, and error‑repair components tailored to nested enterprise schemas. On the DevRev benchmark, the system achieves 91.7% answer correctness, outperforming the next‑best baseline by 54.6 percentage points, and remains competitive on the Spider 2.0 Snowflake dataset.

arXiv AI
Jun 16

VeriGraph: Towards Verifiable Data-Analytic Agents

arXiv:2606. 16603v1 Announce Type: cross Abstract: LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable: a reliance on linear text trajectories makes their reasoning difficult to audit.

By Jiajie Jin, Zhao Yang, Wenle Liao, Yuyang Hu, Guanting Dong, Xiaoxi Li, Yutao Zhu, Zhicheng Dou
arXiv Computation and Language
Sep 18

Schema-Anchored Latent Reasoning for Semantic Parsing-Based Knowledge Base Question Answering

The paper introduces SALR, a schema‑anchored latent reasoning approach for generating logical forms in knowledge‑base question answering. SALR delays explicit schema commitments by generating continuous thoughts in hidden states and aligns these thoughts with a codebook of KB schema elements, guided by an alignment objective derived from gold logical forms. Experiments on GrailQA and WebQSP demonstrate that SALR consistently outperforms strong baselines, notably improving compositional question performance by 2.86 F1 points over TIARA.

By Guangze Gao, Zixuan Li, Sikui Zhang, Chunfeng Yuan, Wenjuan Li, Bing Li, Xiaolong Jin, Weiming Hu
arXiv AI
Aug 7

Tytan: Interactive Neurosymbolic Construction of Analytic Semantic Schemas from Relational Data

arXiv:2608. 06331v1 Announce Type: cross Abstract: From natural-language query interfaces to automated report generation, data analysis tools need a description of the data: the real-world entities it contains, which columns function as measures or identifiers, and how tables connect into units of analysis.

By Donna Hooshmand, Shubham Shahi, Cameron Barrie, Abhratanu Dutta, Marko Sterbentz, Harper Pack, Kristian J. Hammond
arXiv Computation and Language
Sep 25

CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models

CONSISTRE is a consistency‑aware framework for document‑level relation extraction that tackles contradictions in large language model predictions. It offers two tracks: an inference‑time track that refines black‑box LLM outputs through constraint‑aware prompting, verification, and self‑reflection, and a training‑time track that distills consistency knowledge into smaller open‑source models via supervised fine‑tuning and reinforcement learning. Experiments on DocRED show both tracks outperform baselines, with the inference‑time track matching competitive F1 scores and the training‑time track narrowing the performance gap to proprietary LLMs while reducing inference cost.

By Mingxuan Sun
arXiv AI
Aug 26

Constraint-Guided Enterprise Data Mapping with Large Language Models

The paper introduces Constraint‑Guided Enterprise Data Mapping (CGM), a neuro‑symbolic approach that uses schema‑grounded admissibility constraints to steer large language models (LLMs) in aligning enterprise data. CGM operates in three stages: defining constraints with metadata, generating candidates under relaxed constraints to ensure feasibility, and ranking them with a bounded LLM. Experiments show that hard constraints dramatically reduce candidate space and improve F1 scores, enabling small models to match or surpass large LLMs at a fraction of the cost while reducing expert effort.

By Sebastian Monka, Pramod Anantharam, Thien Vo Minh, Lavdim Halilaj