arXiv Machine Learning June 10, 2026 By Haji Gul, Ajaz Ahmad Bhat

When Metrics Disagree: A Meta-Analysis of Knowledge-Graph-Completion Model Benchmarking

Read the original on arXiv Machine Learning →

arXiv:2606. 10287v1 Announce Type: new Abstract: Evaluating Knowledge Graph Completion (KGC) models remains challenging because standard assessment relies on isolated rank-based metrics such as MRR, Hits$@$k, and Mean Rank, which often produce conflicting model orderings across datasets.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

benchmarks

arXiv Machine Learning

Jun 9

Generalized Rank-based Evaluation for Knowledge Graph Completion: Perspectives, Framework, and Analyses

arXiv:2606. 08921v1 Announce Type: new Abstract: Knowledge graph completion (KGC) aims to predict missing facts from an observed knowledge graph (KG), playing a crucial role in a wide range of real-world applications such as drug discovery, recommender systems, and retrieval-augmented generation (RAG).

By Sooho Moon, Jian Kang, Yunyong Ko

llms rag benchmarks safety

arXiv Machine Learning

Jun 9

PROBE-Web: An Interactive System for Probing Evaluation Landscapes of Knowledge Graph Completion Models

arXiv:2606. 08926v1 Announce Type: new Abstract: Knowledge graph completion (KGC) models are commonly evaluated using rank-based metrics such as MRR and Hits@K, despite different users often requiring different evaluation perspectives.

By Sooho Moon, Yunyong Ko

safety

arXiv AI

Jul 7

Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity

arXiv:2604. 24827v2 Announce Type: replace-cross Abstract: Closed-source frontier labs do not disclose parameter counts.

By Bojie Li

llms benchmarks safety

arXiv AI

Jul 14

Coresets Before Score Sets: Evaluation-Unsupervised Prompt Subset Selection for LLM Benchmarks

arXiv:2607. 09739v1 Announce Type: new Abstract: We study LLM benchmark coreset selection: selecting a small subset of prompts over multiple benchmarks whose induced model scores and rankings approximate those obtained from the full benchmark suite.

By Jihan Yao, Gantavya Bhatt, Arnav Das, Peter Jin, Ke Bao, Qiaolin Yu, Khushi Bhardwaj, Chang Su, Jialei Wang, Yikai Zhu, Sugam Devare, Damon Mosk-Aoyama, Zhen Dong, Venkat Krishna Srinivasan, Yineng Zhang, Oleksii Kuchaiev, Jiantao Jiao, Banghua Zhu, Jeff Bilmes

llms rag benchmarks

arXiv AI

Jun 10

KG-SoftMAP: Soft Knowledge-Graph Priors for Bayesian Network Structure Learning from Sparse Discrete Data

arXiv:2606. 10358v1 Announce Type: cross Abstract: Learning Bayesian network (BN) structure from sparse discrete data is hard: when each instance records only a few variables, most variable pairs lack the joint observations needed for reliable scoring, and data-only methods recover little structure.

By Guoliang Xu, James E. Corter

llms benchmarks

arXiv AI

Jul 14

KGCQual: An Interpretable Framework for Evaluating the Knowledge Graph Construction Quality from Text

arXiv:2607. 10212v1 Announce Type: new Abstract: Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance.

By Nipun Misra, Vikranth Udandarao, Aanchal Gupta, Yogender Kumar, Manuj Mukherjee, Raghava Mutharaju

benchmarks safety

Related stories

Generalized Rank-based Evaluation for Knowledge Graph Completion: Perspectives, Framework, and Analyses

PROBE-Web: An Interactive System for Probing Evaluation Landscapes of Knowledge Graph Completion Models

Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity

Coresets Before Score Sets: Evaluation-Unsupervised Prompt Subset Selection for LLM Benchmarks

KG-SoftMAP: Soft Knowledge-Graph Priors for Bayesian Network Structure Learning from Sparse Discrete Data

KGCQual: An Interpretable Framework for Evaluating the Knowledge Graph Construction Quality from Text