arXiv:2602. 02834v4 Announce Type: replace-cross Abstract: What structural inductive bias helps transformers reason over knowledge graphs?
arXiv:2608. 06834v1 Announce Type: new Abstract: Transformers provide a powerful architecture for global content-based matching, but reasoning problems may benefit from a stronger inductive bias toward iterative traversal of latent relations.
PEARL is a new framework for inductive knowledge graph completion that treats relational paths as context-conditioned reasoning signals. It builds a query‑specific contextual subgraph from the query entities’ neighborhoods and uses a large language model‑guided retriever to select semantically relevant paths. By constructing a bipartite interaction graph over paths, contextual entities, and a global subgraph representation, and applying a dual‑view contrastive objective, PEARL adapts path embeddings to local and global structural evidence, achieving the best average Hits@10 on WN18RR, FB15k‑237, and NELL‑995.
The paper introduces GONE, a benchmark for evaluating knowledge unlearning in large language models using structured knowledge graphs, and presents Neighborhood-Expanded Distribution Shaping (NEDS), a framework that leverages graph connectivity to separate forgotten facts from their semantic neighborhood. GONE disentangles direct fact removal, reasoning-based leakage, and catastrophic forgetting, while NEDS achieves high unlearning efficacy and locality on LLaMA-3-8B and Mistral-7B. The dataset is publicly available on Hugging Face.
arXiv:2606. 27967v1 Announce Type: new Abstract: Real-world knowledge graphs are often incomplete, lacking many valid facts.
arXiv:2609. 20278v1 Announce Type: cross Abstract: Text, knowledge graphs, and hypergraphs all have elements that play distinct roles within relation instances, structure that is lost when data is flattened into token sequences.
arXiv:2607. 07422v1 Announce Type: new Abstract: Logical Multi-Hop Query Answering over Knowledge Graphs (KGs) can be formulated as querying, with an implicit completeness assumption.
The paper introduces the Structure-Internalized Rule Language Model (SIRLM) to improve Knowledge Graph Reasoning (KGR) by addressing the mismatch between KG structural context and Large Language Model (LLM) parametric knowledge. SIRLM centers on a Structure-Internalized Rule Generator (SIRG) that uses in-context learning, a structural relation memory, a KG tokenizer, and a neuro-symbolic reasoner to generate structural rules and provide faithful rule-execution feedback. Experiments on 36 datasets against 17 state‑of‑the‑art KGR methods show that SIRLM achieves significant performance gains.
arXiv:2601.10485v5 Announce Type: replace Abstract: Domain-specific knowledge graphs (DKGs) are critical yet often suffer from limited coverage compared to General Knowledge Graphs (GKGs). Existing t...
Large language models can solve complex multi‑hop tasks but often fail on simple two‑hop queries, even when each hop is individually correct. In a controlled symbolic setting, the authors find that models generalize reliably when the second hop follows the training distribution, but always fail when it deviates. Mechanistic analysis shows that successful generalization relies on consistent intermediate representations across contexts, whereas failures arise from a mismatch between lower‑layer representation construction and upper‑layer mapping to outputs. The study proposes a recurrent‑style training strategy that improves out‑of‑distribution two‑hop generalization.
arXiv:2607. 11875v1 Announce Type: cross Abstract: We present a theoretical framework to explain the emergence of inductive reasoning abilities in Transformer language models.
arXiv:2509. 24653v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) excel at multi-hop reasoning in distribution, yet fail on unseen compositions, a phenomenon known as the curse of two-hop reasoning.
FloydNet introduces a learning paradigm that maintains ordered pair states and updates target relations by attending over candidate pairs, inspired by the Floyd–Warshall algorithm. The Pivotal Attention mechanism learns relation composition and pivot weighting in parallel, extending to k-tuples in the ζNet framework. Experiments show ζNet achieves high accuracy on CLRS-30 and near-optimal performance on non-metric TSP instances, matching the discriminative power of k-FWL on BREC.