← Back to all news
arXiv Computation and Language By Seunghui Jwa, Minsu Oh, Sangwoo Hong, Yeo Chan Yoon

Less but Sufficient: Relation Knowledge Selects Compact Knowledge-Graph Contexts, and What External Checks Can and Cannot Catch

Read the original on arXiv Computation and Language →

The paper presents a method that uses relation knowledge—either manually crafted or learned from training data—to select a compact set of knowledge‑graph facts for large language models (LLMs). It introduces derivability, consistency, and validation checks to determine whether an answer can be derived from the selected facts, whether the facts conflict, and whether the derived answer should replace the LLM’s output. Experiments show that seven open LLMs perform better with the compact context than with relevance‑ranked facts, achieving higher exact‑match accuracy on ComplexWebQuestions and outperforming SubgraphRAG across 53 LLM, benchmark, and prompt settings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Oct 27

Knowing When the Evidence Is Enough: Requirement-Directed Search with Horn-Closure Certificates

The paper introduces Certify, a system that determines whether retrieved evidence is sufficient for answering a query by using a requirement-directed search guided by a per-query hypergraph of facts. It decomposes the question, searches for each requirement, verifies entity links, and constructs a Horn-closure certificate that either provides minimal sufficient passages or indicates missing requirements. Experiments on three corpora show that Certify outperforms existing search agents and sufficiency judges, achieving higher decision accuracy and more correct answers.

By Xinyu Zhang, Weiwei Sun
agents
More like this →
arXiv AI
2d ago

Reasoning over Non-Parametric Memory in Multilingual LLMs: A Cognitive-Based Analysis of Relational Knowledge

The paper introduces “Rel-Bloom,” a multilingual diagnostic benchmark of 450 multiple-choice questions that tests language models’ ability to reason over relational facts supplied directly in prompts. It evaluates 43 models across six languages and two multilingual settings, revealing that accuracy drops sharply from Apply to Analyze levels, that Evaluate items remain a bottleneck even for high-performing models, and that difficulty rankings are consistent across languages. The study also finds that counterfactual contexts are generally harder than factual ones, and that removing entity identities can sometimes improve higher-level performance.

By Zohreh Abbasi Liasi, Batoul Hammoud, Anas Madkoor, Ehsaneddin Asgari
llmsragbenchmarks
More like this →
arXiv AI
Aug 11

KGCache: Amortized Subgraph Retrieval for KG Reasoning with LLMs

arXiv:2608. 07954v1 Announce Type: new Abstract: Large language models can answer knowledge-intensive questions more reliably when they are grounded with knowledge graphs, but systems such as Think-on-Graph and Reasoning-on-Graph repeatedly query the same graph neighborhoods across different questions.

By Uros Stanic, Changcheng Yuan, Sabuj Laskar, Ariful Azad
llmsnlp
More like this →
arXiv Computation and Language
Sep 10

The Answer Path and the Grounding Instruction in LLM Question Answering over Knowledge Graphs

The paper investigates how different components of a graph retrieval‑augmented generation pipeline affect large language model performance on knowledge‑graph question answering. It examines four variables—whether the answer path is included, the syntax of triples, the order of triples, and the subgraph size—across six LLMs and two benchmarks. The study finds that including the answer path is crucial, while the grounding instruction dramatically reduces accuracy when no facts are provided, and that syntax, order, and subgraph size have negligible measurable impact at multi‑hop depth.

By Arquimedes Canedo
llmsragnlpbenchmarks
More like this →