← Back to all news
arXiv Computation and Language

Less but Sufficient: Relation Knowledge Selects Compact Knowledge-Graph Contexts, and What External Checks Can and Cannot Catch

The paper presents a method that uses relation knowledge—either manually crafted or learned from training data—to select a compact set of knowledge‑graph facts for large language models (LLMs). It introduces derivability, consistency, and validation checks to determine whether an answer can be derived from the selected facts, whether the facts conflict, and whether the derived answer should replace the LLM’s output. Experiments show that seven open LLMs perform better with the compact context than with relevance‑ranked facts, achieving higher exact‑match accuracy on ComplexWebQuestions and outperforming SubgraphRAG across 53 LLM, benchmark, and prompt settings.

Related stories

arXiv AI
Oct 27

Knowing When the Evidence Is Enough: Requirement-Directed Search with Horn-Closure Certificates

The paper introduces Certify, a system that determines whether retrieved evidence is sufficient for answering a query by using a requirement-directed search guided by a per-query hypergraph of facts. It decomposes the question, searches for each requirement, verifies entity links, and constructs a Horn-closure certificate that either provides minimal sufficient passages or indicates missing requirements. Experiments on three corpora show that Certify outperforms existing search agents and sufficiency judges, achieving higher decision accuracy and more correct answers.

By Xinyu Zhang, Weiwei Sun
agents
More like this →
arXiv AI
2d ago

Reasoning over Non-Parametric Memory in Multilingual LLMs: A Cognitive-Based Analysis of Relational Knowledge

The paper introduces “Rel-Bloom,” a multilingual diagnostic benchmark of 450 multiple-choice questions that tests language models’ ability to reason over relational facts supplied directly in prompts. It evaluates 43 models across six languages and two multilingual settings, revealing that accuracy drops sharply from Apply to Analyze levels, that Evaluate items remain a bottleneck even for high-performing models, and that difficulty rankings are consistent across languages. The study also finds that counterfactual contexts are generally harder than factual ones, and that removing entity identities can sometimes improve higher-level performance.

By Zohreh Abbasi Liasi, Batoul Hammoud, Anas Madkoor, Ehsaneddin Asgari
llmsragbenchmarks
More like this →
arXiv AI
Aug 11

KGCache: Amortized Subgraph Retrieval for KG Reasoning with LLMs

arXiv:2608. 07954v1 Announce Type: new Abstract: Large language models can answer knowledge-intensive questions more reliably when they are grounded with knowledge graphs, but systems such as Think-on-Graph and Reasoning-on-Graph repeatedly query the same graph neighborhoods across different questions.

By Uros Stanic, Changcheng Yuan, Sabuj Laskar, Ariful Azad
llmsnlp
More like this →
arXiv Computation and Language
Sep 10

The Answer Path and the Grounding Instruction in LLM Question Answering over Knowledge Graphs

The paper investigates how different components of a graph retrieval‑augmented generation pipeline affect large language model performance on knowledge‑graph question answering. It examines four variables—whether the answer path is included, the syntax of triples, the order of triples, and the subgraph size—across six LLMs and two benchmarks. The study finds that including the answer path is crucial, while the grounding instruction dramatically reduces accuracy when no facts are provided, and that syntax, order, and subgraph size have negligible measurable impact at multi‑hop depth.

By Arquimedes Canedo
llmsragnlpbenchmarks
More like this →
arXiv AI
Sep 15

FedV-KGQA in Practice: Design Lessons and an Interactive Prototype

FedV-KGQA addresses multi‑hop question answering over vertically partitioned knowledge graphs where each silo holds disjoint relation types. The system trains local embeddings, concatenates silo‑specific entity views, anchors questions at a topic entity, and ranks candidates without sharing raw triples. Experiments show federated fusion nearly matches centralized accuracy, that anchoring and enrichment are more critical than embedding choice, and that the cheapest encoder depends on target accuracy.

By Md Saikat Islam Khan Bappy, Oshani Seneviratne
ragnlp
More like this →
Hugging Face Trending Papers
Jul 13

RAGU: A Multi-Step GraphRAG Engine with a Compact Domain-Adapted LLM

Graph retrieval-augmented generation (GraphRAG) enhances large language models with structured knowledge, yet existing systems construct knowledge graphs in a single extraction pass, producing noisy entities and brittle retrieval. RAGU, an open-source modular GraphRAG engine, addresses this by separating extraction from consolidation: entities and relations pass through two-stage typed extraction, DBSCAN-backed deduplication, LLM summarization, and Leiden community detection.

llmsragnlp
More like this →