← Back to all news
arXiv AI By Yang Fei, Yangfan Jiang, Yin Yang, Xiaokui Xiao

Benchmarking Text-to-SQL under Role-Based Access Control

Read the original on arXiv AI →

The paper introduces a new text‑to‑SQL benchmarking framework that incorporates realistic role‑based access control (RBAC) constraints. It augments existing benchmarks by generating plausible user roles and access policies through an LLM‑assisted workflow, followed by human‑in‑the‑loop quality control. The framework also provides evaluation metrics to detect RBAC‑specific failures and separate SQL utility from compliance, revealing that many high‑scoring models degrade sharply under access constraints.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 18

Effective and Efficient Threat Hunting with Small Language Models

The paper presents a framework for translating natural‑language queries into Kusto Query Language (KQL) using small language models (SLMs). It introduces lightweight retrieval, error‑aware prompting, LoRA fine‑tuning with rationale distillation, and a two‑stage architecture that pairs an SLM drafter with a low‑cost LLM judge. Evaluations on Microsoft’s NL2KQL Defender dataset show the two‑stage approach achieving high syntax and schema‑valid accuracy while dramatically reducing cost compared to larger LLM baselines.

By Saleha Muzammil, Rahul Reddy, Vishal Kamalakrishnan, Hadi Ahmadi, Wajih Ul Hassan
llmsfine-tuningefficiency
More like this →
arXiv AI
Aug 7

Tytan: Interactive Neurosymbolic Construction of Analytic Semantic Schemas from Relational Data

arXiv:2608. 06331v1 Announce Type: cross Abstract: From natural-language query interfaces to automated report generation, data analysis tools need a description of the data: the real-world entities it contains, which columns function as measures or identifiers, and how tables connect into units of analysis.

By Donna Hooshmand, Shubham Shahi, Cameron Barrie, Abhratanu Dutta, Marko Sterbentz, Harper Pack, Kristian J. Hammond
llmsbenchmarks
More like this →