The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper evaluates recent large language models (LLMs) for knowledge graph‑to‑text generation against traditional fine‑tuned encoder‑decoder models. Using both reference‑based metrics and graph‑grounded, reference‑free LLM‑based evaluation, the study finds that LLMs largely close the performance gap on reference metrics and outperform fine‑tuned models on graph‑grounded metrics. The results also show that LLM‑based evaluation correlates weakly with reference metrics, indicating that each captures distinct aspects of generation quality, and that prompted LLMs produce fewer contradictions and omissions, especially as graph size grows.
The paper evaluates recent large language models (LLMs) for knowledge graph-to-text generation against traditional fine‑tuned encoder‑decoder models. Using both reference‑based metrics and graph‑grounded, reference‑free LLM‑based evaluation, the authors find that LLMs close the performance gap on reference metrics and outperform fine‑tuned models on graph‑grounded metrics. The study also shows that LLM‑based evaluation scores correlate poorly with reference metrics, indicating they assess different aspects of quality, and that prompted LLMs produce fewer contradictions and omissions, especially as graph size increases.
arXiv:2607. 10212v1 Announce Type: new Abstract: Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance.
Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance. Existing evaluation practices rely heavily on task-specific metrics or small-scale manual verification, offering limited insight into the structural and semantic fidelity of extracted graphs.
arXiv:2511.04473v3 Announce Type: replace Abstract: Retrieval of information from graph-structured knowledge bases represents a promising direction for improving the factuality of LLMs. While various...
arXiv:2606. 08926v1 Announce Type: new Abstract: Knowledge graph completion (KGC) models are commonly evaluated using rank-based metrics such as MRR and Hits@K, despite different users often requiring different evaluation perspectives.