arXiv Computation and Language

Are Large Language Models Effective Knowledge Graph Constructors?

The paper investigates whether large language models (LLMs) can construct knowledge graphs (KGs) from documents in a zero‑shot, schema‑free manner. It introduces the Detail‑to‑Abstract Hierarchical Knowledge Graph (D2A‑HKG) framework, which splits KG construction into extraction, splitting, and abstraction stages, and evaluates the resulting graphs semantically and structurally. Using seven leading LLMs, the authors benchmark zero‑shot KG construction on CMW‑Lit—a dataset of pediatric mental‑wellness research—and find that state‑of‑the‑art LLMs generally produce relevant, document‑faithful triples with limited hallucination, though their extraction behaviors vary across stages. The study releases CMW‑Lit and the generated graphs as resources for future research and downstream knowledge‑intensive applications.

arXiv AI
Jul 14

KGCQual: An Interpretable Framework for Evaluating the Knowledge Graph Construction Quality from Text

arXiv:2607. 10212v1 Announce Type: new Abstract: Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance.

By Nipun Misra, Vikranth Udandarao, Aanchal Gupta, Yogender Kumar, Manuj Mukherjee, Raghava Mutharaju
Hugging Face Trending Papers
Jul 11

KGCQual: An Interpretable Framework for Evaluating the Knowledge Graph Construction Quality from Text

Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance. Existing evaluation practices rely heavily on task-specific metrics or small-scale manual verification, offering limited insight into the structural and semantic fidelity of extracted graphs.

arXiv Computation and Language
Oct 23

An Empirical Study of Recent Large Language Models for Knowledge Graph to Text Generation on Standard Benchmarks

The paper evaluates recent large language models (LLMs) for knowledge graph-to-text generation against traditional fine‑tuned encoder‑decoder models. Using both reference‑based metrics and graph‑grounded, reference‑free LLM‑based evaluation, the authors find that LLMs close the performance gap on reference metrics and outperform fine‑tuned models on graph‑grounded metrics. The study also shows that LLM‑based evaluation scores correlate poorly with reference metrics, indicating they assess different aspects of quality, and that prompted LLMs produce fewer contradictions and omissions, especially as graph size increases.

By Priyanka, Renaud Delbru, Souhail Bakkali, Micka\"el Coustaty, Antoine Doucet
Hugging Face Trending Papers
Oct 22

An Empirical Study of Recent Large Language Models for Knowledge Graph to Text Generation on Standard Benchmarks

The paper evaluates recent large language models (LLMs) for knowledge graph‑to‑text generation against traditional fine‑tuned encoder‑decoder models. Using both reference‑based metrics and graph‑grounded, reference‑free LLM‑based evaluation, the study finds that LLMs largely close the performance gap on reference metrics and outperform fine‑tuned models on graph‑grounded metrics. The results also show that LLM‑based evaluation correlates weakly with reference metrics, indicating that each captures distinct aspects of generation quality, and that prompted LLMs produce fewer contradictions and omissions, especially as graph size grows.

arXiv AI
Sep 2

Automated Tree Knowledge Graph Construction using Ontology Expansion and Retrieval from Vietnamese History Textbooks

The paper presents an end‑to‑end pipeline for automatically constructing a tree‑structured knowledge graph (KG) from Vietnamese high school history textbooks and evaluating retrieval strategies that exploit the KG’s hierarchical structure. The KG construction uses a three‑phase hybrid relation extraction process, including intra‑batch deduplication, approximate cross‑batch search, and LLM extraction with a centroid filter and dual‑LLM validator, resulting in 750 nodes and 4,341 semantic edges across 41 ontology types. Retrieval evaluation compares three graph traversal strategies—Top‑Down, Horizontal, and Bottom‑Up—on a benchmark of 1,210 Vietnamese queries, finding that the Top‑Down strategy with structural information outperforms a vector baseline by 4.7 percentage points in NDCG@10.

By Ket Doan Nguyen, Minh N. H. Nguyen
arXiv AI
Jun 17

A Unified Framework for Context-Aware and Relation-Aware Graph Retrieval-Augmented Generation

arXiv:2606. 18075v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) has emerged as a paradigm for enhancing large language models (LLMs) with external knowledge, yet existing graph-based methods face a fundamental limitation: entity-centric and chunk-centric approaches operate on representations anchored to original text without true knowledge fusion.

By Haoyang Zhong, Yifei Sun, Antong Zhang, Chunping Wang, Lei Chen, Yang Yang
arXiv Computation and Language
Aug 25

ConvergeWriter: Data-Driven Bottom-Up Article Construction

ConvergeWriter introduces a bottom‑up, data‑driven framework for long‑form document generation that first retrieves exhaustive knowledge from a source corpus and clusters it into distinct knowledge groups. These clusters then guide the creation of a hierarchical outline and the final text, ensuring the output is strictly grounded in the retrieved material and traceable to its sources. Experiments on 14B and 32B LLMs show that this approach matches or surpasses state‑of‑the‑art baselines, especially in scenarios requiring high factual fidelity and structural coherence.

By Binquan Ji, Jiaqi Wang, Ruiting Li, Xingchen Han, Yiyang Qi, Shichao Wang, Yifei Lu, Yuantao Han, Feiliang Ren