arXiv AI

Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation

The paper introduces ParallaxRAG, a multi-view retrieval-augmented generation framework that addresses multi-hop reasoning challenges in large language models over knowledge graphs. It identifies that transformer attention heads specialize in distinct semantic relations across reasoning stages, forming a hop-aligned relay pattern, and proposes decoupling queries and KGs into head-specific semantic spaces to enforce relational diversity. Experiments on WebQSP, CWQ, and BioASQ show state‑of‑the‑art performance, reduced hallucination, and strong generalization.

arXiv AI

Oct 9

GLM-RAG: Graph Language Models for Graph-Based Retrieval-Augmented Generation

GLM-RAG introduces a graph language model (GLM) based retriever for retrieval‑augmented generation over knowledge graphs. The study compares GLM‑based, graph neural network (GNN)‑based, and vector‑search retrievers across single‑ and multi‑hop settings, emphasizing transferability to unseen domains. Results show that finetuned GLM retrievers achieve state‑of‑the‑art performance on two multi‑hop benchmarks and generalize better out of domain, while GNN retrievers excel at graph coverage and vector‑search performs best on single‑hop tasks.

By Maya Arseven, Anette Frank, Beni Egressy, Johann Higl, Moritz Plenz

llms rag benchmarks

arXiv AI

Oct 27

UniGFM-RAG: Unify Graph Foundation Models and Large Language Models for End-to-End Retrieval and Reasoning

arXiv:2610.34866v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown remarkable capabilities across tasks, but they still struggle with complex reasoning that requires integrating...

By Zicheng Zhao, Linhao Luo, Zongji Yu, Junnan Dong, Yanchao Tan, Shirui Pan, Chen Gong

llms rag efficiency benchmarks

arXiv AI

Jul 2

DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning

arXiv:2607. 00341v1 Announce Type: cross Abstract: Large language models achieve strong performance on many reasoning tasks when allowed to externalize intermediate steps as Chain-of-Thought (CoT).

By Hengyu Fu, Tianyu Guo, Zixuan Wang, Hanlin Zhu, Jason D. Lee, Jiantao Jiao, Stuart Russell, Song Mei

llms rag benchmarks

arXiv Machine Learning

Sep 30

UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG

arXiv:2603.28773v2 Announce Type: replace-cross Abstract: Large language models (LLMs) frequently generate confident yet factually incorrect content when used for language generation (a phenomenon of...

By Dobrik Georgiev, Kheeran K. Naidu, Alberto Cattaneo, Federico Monti, Carlo Luschi, Daniel Justus

llms rag nlp benchmarks safety

arXiv AI

Jun 17

FlowRAG: Synergizing Explicit Reasoning via Frequency-Aware Multi-Granularity Graph Flow

arXiv:2606. 17856v1 Announce Type: new Abstract: Graph-based retrieval-augmented generation (GraphRAG) is effective for knowledge-intensive and multi-hop query tasks; however, many existing methods primarily seed entity-based graphs and rely on implicit semantic relevance propagation.

By Bihao Zhan, Zongsheng Cao, Jie Zhou, Bo Zhang, Liang He

rag efficiency benchmarks safety

arXiv Computation and Language

Oct 22

Graph Topology Meets Energy-Driven Pathfinding: Preserving Contextual Consistency in Retrieval-Augmented Generation

ThreadRAG is a retrieval‑augmented generation framework that preserves contextual consistency by structuring retrieved documents into Semantic Nodes and building a Semantic Topology Graph (STG). It uses Energy‑Driven Pathfinding (EDP) to identify query‑relevant subgraphs and linearize them into coherent textual evidence paths. Experiments on HotpotQA, 2WikiMultihop, and another benchmark show ThreadRAG outperforms strong baselines, achieving a 14.4% EM gain on HotpotQA and a 5.8% EM lead on 2WikiMultihop while cutting token cost to 23.3% of the strongest baseline.

By Yufeng Li, Zaijing Li, Fuchen Zhang, Zongzong Wu, Ming Zhao, Fengxiao Tang

llms rag diffusion nlp benchmarks

arXiv Computation and Language

Oct 1

Explore-on-Graph: Hybrid Embedding-LLM Reasoning for Knowledge Graph Question Answering under Incompleteness

arXiv:2609.39786v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly combined with knowledge graphs (KGs) to ground reasoning in structured evidence. However, most LLM-based...

By Ola El Khatib, Djellel Difallah

llms rag nlp benchmarks

arXiv AI

Jun 30

Query-Aware Spreading Activation for Multi-Hop Retrieval over Knowledge Graphs

arXiv:2606. 30133v1 Announce Type: cross Abstract: Retrieval-augmented generation built on knowledge graphs (Graph RAG) outperforms flat passage retrieval on multi-hop question answering by leveraging graph structure.

By Illia Makarov, Mykola Glybovets

rag diffusion nlp benchmarks

arXiv AI

Jul 8

RSF-GLLM: Bridging the Semantic Gap in Multi-Hop Knowledge Graph QA via Recurrent Soft-Flow and Decoupled LLM Generation

arXiv:2607. 06527v1 Announce Type: cross Abstract: Multi-hop Question Answering over Knowledge Graphs faces a critical challenge: traditional retrieve-then-read pipelines break differentiability, preventing the retriever from learning to bridge the semantic gap where intermediate nodes lack lexical overlap with the query.

By Sambaran Bandyopadhyay, Ananth Muppidi

llms nlp fine-tuning efficiency

arXiv Computation and Language

Sep 14

Cognition on Graph: Navigating Massive Knowledge Space via Cognitive Cycles and Bidirectional Graph-Text Synergy

arXiv:2609.12791v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) has empowered Large Language Models (LLMs) to tackle knowledge-intensive tasks. However, navigating global, hetero...

By Gengxian Zhou, Jian Xu, Zichen Tang, Shiming Xiang, Haihong E, Cheng-Lin Liu

llms rag benchmarks

arXiv Computation and Language

Oct 22

MARS: Multi-hop Adaptive Retrieval and SPARQL Generation for KGQA

MARS is a scalable knowledge‑graph question‑answering system that integrates large language models with knowledge graphs without requiring model fine‑tuning. It uses a structured retrieval process that links question entities to the graph, iteratively fetching next‑hop information and deciding when to stop and generate a SPARQL query. Evaluated on three KGQA benchmarks—including multilingual settings—MARS matches state‑of‑the‑art performance while remaining efficient and scalable.

By Nikit Srivastava, Daniel Vollmers, Ren\'e Speck, Nikolaos Karalis, Hamada M. Zahera, Axel-Cyrille Ngonga Ngomo

llms agents nlp fine-tuning benchmarks

arXiv AI

Jun 30

SemFlowRAG: Directed Semantic Flow from Abstraction to Evidence for Complex Reasoning

arXiv:2606. 28447v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) enhanced by Knowledge Graphs has shown promise in complex multi-hop reasoning tasks.

By Houyuan Qin, Rong Wu, Qinyuan Qin, Botian Shi, Jingjing Qu, Yang Sun, Pinlong Cai

rag

Related stories