arXiv AI August 28, 2026 By Ahlame Diouan (ERIC, UL2), Eric Ferey (ERIC, UL2), Sabine Loudcher (ERIC, UL2), J\'er\^ome Darmont (ERIC, UL2)

Discovering Relationships in Data Lakes Using Large Language Models: An Industrial Case

Read the original on arXiv AI →

The paper introduces ColRel, a two-stage approach that uses large language models to discover relationships between columns in data lakes. It first creates column embeddings from available metadata and data at ingestion, then refines these embeddings with business dictionaries to generate concise natural-language descriptions. Experiments on public benchmarks and an industrial ERP dataset demonstrate ColRel’s effectiveness, especially in scenarios with weak signals and semantically related columns.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Aug 27

Discovering Relationships in Data Lakes Using Large Language Models: An Industrial Case

Data lakes rely on metadata to remain usable, yet this meta data is often limited or weakly informative for column relationship discovery, especially in ERP-derived datasets with coded or abbreviated...

llms rag benchmarks

arXiv AI

1d ago

TabJoinBench: A Benchmark for Joinable Table Discovery

arXiv:2610.00817v1 Announce Type: cross Abstract: Join discovery aims to identify tables from large data repositories that can augment a query table with complementary information, enabling downstrea...

By Sandipan De, Jin Wang, Vivek Gupta

rag benchmarks

arXiv AI

Jun 30

Database Context Compression for Text-to-SQL on Real-World Large Databases

arXiv:2606. 28601v1 Announce Type: cross Abstract: Recent progress in Text-to-SQL has been driven by stronger language models and prompting strategies, yet performance on real enterprise benchmarks such as Spider 2.

By Jingwen Liu, Weibin Liao, Xin Gao, Junfeng Zhao, Yasha Wang

llms benchmarks data-engineering

arXiv Computation and Language

Sep 16

SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP

SciNLP is a new benchmark dataset for full‑text entity and relation extraction in the NLP domain, comprising 60 manually annotated papers with 6,429 entities and 1,649 relations. It is the first dataset to provide full‑text annotations of entities and their relationships specifically for NLP literature. Experiments show that models trained on SciNLP outperform baselines on certain tasks, and the dataset enabled the automatic construction of a fine‑grained knowledge graph with an average node degree of 3.3.

By Decheng Duan, Yingyi Zhang, Jitong Peng, Chengzhi Zhang

nlp benchmarks

arXiv Machine Learning

Sep 23

Discovery-Driven Integration of Disjoint Tables via Text

arXiv:2609.26658v1 Announce Type: cross Abstract: Integrating heterogeneous datasets within data lakes is a critical challenge, particularly for semantically related tables that lack the explicit att...

By Md Ataur Rahman, Dimitris Sacharidis, Oscar Romero, Sergi Nadal

llms multimodal benchmarks

arXiv AI

Jun 2

APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQL

arXiv:2602. 16720v2 Announce Type: replace-cross Abstract: Text-to-SQL systems powered by Large Language Models have excelled on academic benchmarks but struggle in complex enterprise environments.

By Bowen Cao, Weibin Liao, Yushi Sun, Dong Fang, Haitao Li, Wai Lam

llms agents efficiency benchmarks data-engineering

One email a morning, machine-written

Related stories

Discovering Relationships in Data Lakes Using Large Language Models: An Industrial Case

TabJoinBench: A Benchmark for Joinable Table Discovery

Database Context Compression for Text-to-SQL on Real-World Large Databases

SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP

Discovery-Driven Integration of Disjoint Tables via Text

APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQL