arXiv Machine Learning

Enabling Knowledge Graph Understanding at Scale with the EXplore Your Graphs ENgine (EXYGEN)

The paper introduces EXYGEN, a framework that enables conversational access to large knowledge graphs by combining VoID descriptions, ShEx schemas, retrieved triples, and example question‑query pairs in a retrieval‑augmented generation pipeline. On the SciQA benchmark, this approach achieves an exact‑match score of 0.419 without fine‑tuning any large language model, and shows that larger general‑purpose LLMs can outperform smaller code‑specialized ones when provided sufficient context. To scale metadata generation for very large KGs, the authors propose a predicate‑coverage‑aware parallel graph sampling strategy that preserves structural diversity, reduces runtime by over 80× on OpenCitations Meta and GESIS, and is the only tractable method for obtaining complete metadata on ORKG.

arXiv AI

Sep 25

SPARQL-LLM: Real-Time SPARQL Query Generation from Natural Language Questions

SPARQL-LLM is an open‑source, triplestore‑agnostic system that generates SPARQL queries from natural language using lightweight metadata and dedicated components for indexing, prompt building, and execution. It achieves up to 59 % higher F1 scores than the next best system on a multilingual challenge and on bioinformatics knowledge graphs, while being up to 27 × faster and costing no more than $0.01 per question. The project is publicly available on GitHub and is already deployed on real‑world decentralized knowledge graphs such as expasy.org/chat.

By Panayiotis Smeros, Vincent Emonet, Ruijie Wang, Ana-Claudia Sima, Tarcisio Mendes de Farias

llms benchmarks

arXiv AI

Aug 6

Text2GraphQuery-Bench: A Text to Graph Query Benchmark

arXiv:2602. 11745v2 Announce Type: replace Abstract: Graph models are fundamental to data analysis in domains rich with complex relationships.

By Songlin Lyu, Lujie Ban, Zihang Wu, Tianqi Luo, Jirong Liu, Ayoub Moussaid, Oskar van Rest, Heng Lin, Chenhao Ma, Nan Tang, Shipeng Qi, Yongchao Liu, Zhan Qiu, Juelu Zhang, Jiajun Zheng

llms fine-tuning benchmarks data-engineering

arXiv AI

Jun 2

From Graph Retrieval to Schema Realization: Counterfactual Validation for Text-to-SPARQL over Heterogeneous Knowledge Graphs

arXiv:2508. 01815v2 Announce Type: replace-cross Abstract: Text-to-SPARQL maps natural-language questions to executable SPARQL queries over RDF knowledge graphs.

By Yang Zhao, Chengxiao Dai, Yue Xiu, Dusit Niyato

agents nlp benchmarks safety

arXiv Computation and Language

Sep 1

RENSA: Rich Environment Metadata to Navigate Shared and Distributed Endpoints for Automated Federated SPARQL Query Generation

RENSA is a federated SPARQL query generation framework that extends SPARQL Builder Metadata to include class and authority information, enabling precise source selection and semantic constraint inference without runtime ASK queries. The generated metadata profiles occupy less than 1% of the original dataset triples, providing storage‑efficient insights. Evaluation on the LargeRDFBench benchmark shows that RENSA matches state‑of‑the‑art source selection performance while eliminating runtime communication overhead.

By Victor Eiti Yamamoto, Takeda Hideaki, Yamamoto Yasunori

benchmarks

arXiv AI

Oct 2

Build2SPARQL: A Large-Scale Text-to-SPARQL Benchmark Dataset for Building Knowledge Graph Querying

Build2SPARQL is a large-scale benchmark dataset for translating natural-language questions into SPARQL queries over building knowledge graphs. The dataset is generated by a KG‑grounded pipeline that produces 6,136 executable SPARQL queries and 30,680 corresponding natural-language questions across six query-pattern families and five vocabulary registers, covering 201 building KGs. Human validation shows high semantic fidelity, naturalness, and operational plausibility, and retrieval‑augmented evaluation demonstrates significant accuracy gains for open‑weight language models.

By Wooyoung Jung

llms rag agents benchmarks

arXiv AI

Jul 17

SAGA: Schema-Aware Grounding for Agentic Text-to-SPARQL Generation

arXiv:2607. 14494v1 Announce Type: new Abstract: Complex knowledge base question answering (KBQA) is commonly approached through either information retrieval over a question-specific subgraph or semantic parsing into an executable logical form.

By Yiming Zhang, Koji Tsuda

llms rag agents nlp benchmarks

arXiv Machine Learning

Aug 17

AutoSchema: Live Schema Grounding for Agentic Text-to-Sparql over Heterogeneous Knowledge Graphs

arXiv:2608. 14228v1 Announce Type: new Abstract: Life science knowledge graphs make large collections of structured data available through SPARQL, but each resource uses its own schema, identifiers, and links.

By Yiming Zhang, Koji Tsuda

llms agents

arXiv Computation and Language

Oct 22

MARS: Multi-hop Adaptive Retrieval and SPARQL Generation for KGQA

MARS is a scalable knowledge‑graph question‑answering system that integrates large language models with knowledge graphs without requiring model fine‑tuning. It uses a structured retrieval process that links question entities to the graph, iteratively fetching next‑hop information and deciding when to stop and generate a SPARQL query. Evaluated on three KGQA benchmarks—including multilingual settings—MARS matches state‑of‑the‑art performance while remaining efficient and scalable.

By Nikit Srivastava, Daniel Vollmers, Ren\'e Speck, Nikolaos Karalis, Hamada M. Zahera, Axel-Cyrille Ngonga Ngomo

llms agents nlp fine-tuning benchmarks

arXiv Computation and Language

Oct 7

Wikidata Search Traces: A Dataset for Training Knowledge Graph Search Agents

arXiv:2610.06650v2 Announce Type: replace Abstract: Wikidata is one of the largest open knowledge bases, yet answering a complex question over it still requires a SPARQL query that names the right en...

By Mohamed Chenene, Carlos Rosas-Hinostroza, Anastasia Stasenko, Shani Evenstein Sigalov, Pierre-Carl Langlais

llms agents

arXiv AI

2d ago

UniGFM-RAG: Unify Graph Foundation Models and Large Language Models for End-to-End Retrieval and Reasoning

arXiv:2610.34866v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown remarkable capabilities across tasks, but they still struggle with complex reasoning that requires integrating...

By Zicheng Zhao, Linhao Luo, Zongji Yu, Junnan Dong, Yanchao Tan, Shirui Pan, Chen Gong

llms rag efficiency benchmarks

arXiv AI

Jul 21

Natural Language Access to Domain-Specific Metadata: A Reusable Framework for LLM Query Generation

arXiv:2607. 18029v1 Announce Type: cross Abstract: Researchers need to answer ad-hoc questions about the contents of domain-specific archives but often lack the expertise to write structured queries on the metadata.

By Blake G. Fitch, Cato Elia Kurtz

llms agents fine-tuning benchmarks safety data-engineering

arXiv Machine Learning

Sep 30

UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG

arXiv:2603.28773v2 Announce Type: replace-cross Abstract: Large language models (LLMs) frequently generate confident yet factually incorrect content when used for language generation (a phenomenon of...

By Dobrik Georgiev, Kheeran K. Naidu, Alberto Cattaneo, Federico Monti, Carlo Luschi, Daniel Justus

llms rag nlp benchmarks safety

Related stories