The paper evaluates recent large language models (LLMs) for knowledge graph‑to‑text generation against traditional fine‑tuned encoder‑decoder models. Using both reference‑based metrics and graph‑grounded, reference‑free LLM‑based evaluation, the study finds that LLMs largely close the performance gap on reference metrics and outperform fine‑tuned models on graph‑grounded metrics. The results also show that LLM‑based evaluation correlates weakly with reference metrics, indicating that each captures distinct aspects of generation quality, and that prompted LLMs produce fewer contradictions and omissions, especially as graph size grows.
The paper evaluates recent large language models (LLMs) for knowledge graph-to-text generation against traditional fine‑tuned encoder‑decoder models. Using both reference‑based metrics and graph‑grounded, reference‑free LLM‑based evaluation, the authors find that LLMs close the performance gap on reference metrics and outperform fine‑tuned models on graph‑grounded metrics. The study also shows that LLM‑based evaluation scores correlate poorly with reference metrics, indicating they assess different aspects of quality, and that prompted LLMs produce fewer contradictions and omissions, especially as graph size increases.
By Priyanka, Renaud Delbru, Souhail Bakkali, Micka\"el Coustaty, Antoine Doucet
arXiv:2607. 10212v1 Announce Type: new Abstract: Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance.
By Nipun Misra, Vikranth Udandarao, Aanchal Gupta, Yogender Kumar, Manuj Mukherjee, Raghava Mutharaju
Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance. Existing evaluation practices rely heavily on task-specific metrics or small-scale manual verification, offering limited insight into the structural and semantic fidelity of extracted graphs.
arXiv:2511.04473v3 Announce Type: replace
Abstract: Retrieval of information from graph-structured knowledge bases represents a promising direction for improving the factuality of LLMs. While various...
By Alberto Cattaneo, Carlo Luschi, Daniel Justus
arXiv:2606. 08926v1 Announce Type: new Abstract: Knowledge graph completion (KGC) models are commonly evaluated using rank-based metrics such as MRR and Hits@K, despite different users often requiring different evaluation perspectives.
By Sooho Moon, Yunyong Ko
arXiv:2606. 08921v1 Announce Type: new Abstract: Knowledge graph completion (KGC) aims to predict missing facts from an observed knowledge graph (KG), playing a crucial role in a wide range of real-world applications such as drug discovery, recommender systems, and retrieval-augmented generation (RAG).
By Sooho Moon, Jian Kang, Yunyong Ko
arXiv:2606. 29180v1 Announce Type: new Abstract: A Knowledge Graph (KG) represents facts as structured triples and is widely used to organize relational knowledge across diverse domains.
By Seungryeol Baek, Wooseok Sim, Hogun Park
The paper investigates whether large language models (LLMs) can construct knowledge graphs (KGs) from documents in a zero‑shot, schema‑free manner. It introduces the Detail‑to‑Abstract Hierarchical Knowledge Graph (D2A‑HKG) framework, which splits KG construction into extraction, splitting, and abstraction stages, and evaluates the resulting graphs semantically and structurally. Using seven leading LLMs, the authors benchmark zero‑shot KG construction on CMW‑Lit—a dataset of pediatric mental‑wellness research—and find that state‑of‑the‑art LLMs generally produce relevant, document‑faithful triples with limited hallucination, though their extraction behaviors vary across stages. The study releases CMW‑Lit and the generated graphs as resources for future research and downstream knowledge‑intensive applications.
By Ruirui Chen, Weifeng Jiang, Chengwei Qin, Bo Xiong, Kaiwen Wei, Fiona Liausvia, Pei Fang Tan, Ker Yung Chua, Dongkyu Choi, Mukkesh Kumar, Evelyn C. Law, Dennis Wang, Boon Kiat Quek
arXiv:2607. 25959v1 Announce Type: cross Abstract: Wikipedia and Wikidata are widely used for information access, LLM pre-training, and retrieval-augmented generation.
By Fanfu Wei, Thibault Ehrhart, Rapha\"el Troncy
The paper introduces a knowledge‑graph‑based evaluation framework, S3KG, to assess whether large language models truly understand context in question answering tasks. S3KG combines structural and semantic signals into a single similarity score and is paired with a diagnostic analysis that pinpoints reasoning errors at the triplet level. Across nine benchmarks, the method outperforms existing baselines, achieving up to +7.6 F1 points and an AUROC of 0.973.
By Subavarshana Arumugam, Mamta Nallaretnam, Kithuni Wickramasinghe, Chamath Gunapala, Pragatheeswaran Vipulanandan, Kamal Premaratne, Uthayasanker Thayasivam
arXiv:2606. 03365v1 Announce Type: new Abstract: Embedding models (KGEMs) constitute the main link prediction approach to complete knowledge graphs.
By Guillaume M\'erou\'e, Fabien Gandon, Pierre Monnin