The paper evaluates recent large language models (LLMs) for knowledge graph‑to‑text generation against traditional fine‑tuned encoder‑decoder models. Using both reference‑based metrics and graph‑grounded, reference‑free LLM‑based evaluation, the study finds that LLMs largely close the performance gap on reference metrics and outperform fine‑tuned models on graph‑grounded metrics. The results also show that LLM‑based evaluation correlates weakly with reference metrics, indicating that each captures distinct aspects of generation quality, and that prompted LLMs produce fewer contradictions and omissions, especially as graph size grows.
arXiv:2506. 06331v2 Announce Type: replace-cross Abstract: By retrieving contexts from knowledge graphs, graph-based retrieval-augmented generation (GraphRAG) enhances large language models (LLMs) to generate quality answers for user questions.
By Qiming Zeng, Hao Luo, Yuhao Lin, Yicheng Jin, Yuxiang Wang, Fangcheng Fu, Xiao Yan, Jiawei Jiang
Large Language Models (LLMs) are being tested as a cost‑effective alternative to human judgment for evaluating machine translation (MT). This study compares LLM agreement with human annotators on two MT quality schemes—Multidimensional Quality Metrics (MQM) and Error Span Annotation (ESA)—across 70 language pairs and public WMT23/25 data. Results show that while LLMs sometimes match or exceed human agreement, performance varies widely by scheme, language, and domain, and remains unreliable in many settings. The work also identifies specific challenges for both humans (e.g., fine‑grained MQM, low‑resource languages) and LLMs (e.g., minor errors, wrong language variants, span annotation).
By Hala Almaghout, Christian Federmann, Qin Gao
arXiv:2511.04473v3 Announce Type: replace
Abstract: Retrieval of information from graph-structured knowledge bases represents a promising direction for improving the factuality of LLMs. While various...
By Alberto Cattaneo, Carlo Luschi, Daniel Justus
The paper compares Knowledge-Graph Based Augmentation (Graph-RAG) with Retrieval-Augmented Generation (RAG) for answering culturally specific questions. Using the LatamQA dataset, Graph-RAG, built automatically from Wikipedia via KGGen, matches RAG performance and reduces the base LLM’s error by 72% with a standard KG and 78% with a benchmark-aware variant. The approach also transfers zero‑shot to Portuguese, showing multilingual applicability.
By Pablo Poulenard, Yannis Karmim, Valentin Barri\`ere
arXiv:2607. 10212v1 Announce Type: new Abstract: Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance.
By Nipun Misra, Vikranth Udandarao, Aanchal Gupta, Yogender Kumar, Manuj Mukherjee, Raghava Mutharaju
arXiv:2610.31684v1 Announce Type: new
Abstract: Knowledge Graphs (KGs) have been popularized during the last few decades and are increasingly constructed and consumed by Large Language Models (LLMs)....
By Elwin Huaman
arXiv:2511. 07457v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in modeling sequential textual data and generalizing across diverse tasks.
By Jiarui Feng, Donghong Cai, Yixin Chen, Muhan Zhang
Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance. Existing evaluation practices rely heavily on task-specific metrics or small-scale manual verification, offering limited insight into the structural and semantic fidelity of extracted graphs.
arXiv:2610.19166v1 Announce Type: new
Abstract: Meta-evaluation of text generation metrics and LLM-judges is primarily conducted using metric-human correlations. However, prior work has shown that co...
By Anubhav Jangra, Adrian de Wynter
The paper investigates whether large language models (LLMs) can construct knowledge graphs (KGs) from documents in a zero‑shot, schema‑free manner. It introduces the Detail‑to‑Abstract Hierarchical Knowledge Graph (D2A‑HKG) framework, which splits KG construction into extraction, splitting, and abstraction stages, and evaluates the resulting graphs semantically and structurally. Using seven leading LLMs, the authors benchmark zero‑shot KG construction on CMW‑Lit—a dataset of pediatric mental‑wellness research—and find that state‑of‑the‑art LLMs generally produce relevant, document‑faithful triples with limited hallucination, though their extraction behaviors vary across stages. The study releases CMW‑Lit and the generated graphs as resources for future research and downstream knowledge‑intensive applications.
By Ruirui Chen, Weifeng Jiang, Chengwei Qin, Bo Xiong, Kaiwen Wei, Fiona Liausvia, Pei Fang Tan, Ker Yung Chua, Dongkyu Choi, Mukkesh Kumar, Evelyn C. Law, Dennis Wang, Boon Kiat Quek
arXiv:2508. 10971v2 Announce Type: replace-cross Abstract: Knowledge graphs (KGs) can be enhanced through rule mining; however, the resulting logical rules are often difficult for humans to interpret due to their inherent complexity and the idiosyncratic labeling conventions of individual KGs.
By Nasim Shirvani-Mahdavi, Chengkai Li