arXiv Computation and Language By Xingyu Tan, Xiaoyang Wang, Qing Liu, Xiwei Xu, Xin Yuan, Liming Zhu, Wenjie Zhang

Ctrl-G: Controlling Structural and Interaction Exposure in Knowledge Graph Augmented LLM Reasoning

Read the original on arXiv Computation and Language →

Ctrl‑G is a knowledge‑graph augmented retrieval‑augmented generation (KG‑RAG) framework that jointly limits structural and interaction exposure when sending graph evidence to a remote large language model (LLM). It constructs query‑conditioned sanitized views via structural aggregation and pruning, and uses a hierarchical controller to prune exploration paths while verifying evidence against the original KG. By reusing abstract, verified reasoning traces in local memory, Ctrl‑G reduces repeated disclosure across questions and achieves the highest accuracy on five of six benchmarks, outperforming the strongest baseline by up to 13.8 points.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI
Sep 24

The Path Matters: Evaluating Small Language Models Beyond Answer Accuracy in KGQA

The paper investigates how small language models (SLMs) perform in knowledge graph question answering (KGQA) when evaluated on the reasoning paths they take, rather than just the final answer. Using the THESEUS navigation and traceability framework, the authors test frozen, off‑the‑shelf SLMs as local action policies that choose graph actions and decide when to stop, without any task‑specific training or free‑form answer generation. By measuring both Hits@1 and Path Edit Distance (PED) across the Kinship and MQuAKE‑ST datasets, the study finds that models vary significantly in both answer accuracy and path fidelity, and that prompting can either help or hurt navigation depending on the model. "whyItMatters":"The results show that evaluating SLMs solely on endpoint accuracy can be misleading, highlighting the need to assess reasoning path fidelity in KGQA tasks."

By Eduin E. Hernandez, Sergio A. Diaz, Luis F. Garcia, Nurassyl Askar, Stefano Rini