arXiv Computation and Language

Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds

The paper shows that large language models (LLMs) naturally organize their hidden state manifolds into small‑world networks, enabling efficient multi‑hop reasoning. By converting similarity matrices into unweighted graphs, the authors trace connectivity between distant semantic anchors and find a sharp topological phase transition: deep reasoning layers compress conceptual distances into paths bounded by six semantic hops, while early syntactic layers remain fragmented. The framework is applied to zero‑shot hallucination detection in Retrieval‑Augmented Generation, revealing that factual generations preserve a ~3‑hop structure, whereas hallucinations collapse the topology.

arXiv Computation and Language

2d ago

Graph Topology Meets Energy-Driven Pathfinding: Preserving Contextual Consistency in Retrieval-Augmented Generation

ThreadRAG is a retrieval‑augmented generation framework that preserves contextual consistency by structuring retrieved documents into Semantic Nodes and building a Semantic Topology Graph (STG). It uses Energy‑Driven Pathfinding (EDP) to identify query‑relevant subgraphs and linearize them into coherent textual evidence paths. Experiments on HotpotQA, 2WikiMultihop, and another benchmark show ThreadRAG outperforms strong baselines, achieving a 14.4% EM gain on HotpotQA and a 5.8% EM lead on 2WikiMultihop while cutting token cost to 23.3% of the strongest baseline.

By Yufeng Li, Zaijing Li, Fuchen Zhang, Zongzong Wu, Ming Zhao, Fengxiao Tang

llms rag diffusion nlp benchmarks

arXiv AI

Jun 30

SemFlowRAG: Directed Semantic Flow from Abstraction to Evidence for Complex Reasoning

arXiv:2606. 28447v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) enhanced by Knowledge Graphs has shown promise in complex multi-hop reasoning tasks.

By Houyuan Qin, Rong Wu, Qinyuan Qin, Botian Shi, Jingjing Qu, Yang Sun, Pinlong Cai

rag

arXiv AI

Aug 6

Topology-Aware Reasoning over Incomplete Knowledge Graph with Graph-Based Soft Prompting

arXiv:2604. 12503v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have shown remarkable capabilities across various tasks but remain prone to hallucinations in knowledge-intensive scenarios.

By Shuai Wang, Xixi Wang, Yinan Yu

llms rag nlp benchmarks safety

Hugging Face Trending Papers

Jul 9

Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench

Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them. However, existing long-context evaluations - from Needle-in-a-Haystack (NIAH) tests to more recent multi-hop reasoning and summarization tasks - predominantly measure average-case performance, and many are either saturated or lack robustness.

llms nlp benchmarks

arXiv Computer Vision

Sep 1

TopoAgent: A Structure-Aware Perception-to-Reasoning Framework for Diagram-to-Graph Topology Extraction with Large Vision-Language Models

The paper introduces TopoBench-180, a human‑verified benchmark of 180 structural diagrams with canonical graph annotations, and TopoAgent, a perception‑to‑reasoning framework that extracts graph topology from diagrams using large vision‑language models. TopoAgent combines grounded perception, global structural priors, node inventory construction, local‑to‑global relation reasoning, and consistency enforcement to progressively build the target graph. Experiments demonstrate that TopoAgent surpasses strong baselines, particularly in edge extraction, thereby advancing multimodal structured understanding for diagram‑to‑graph tasks.

By Bangwei Guo, Xujiang Zhao, Yanchi Liu, Wei Cheng, Shengyu Chen, Dongyue Li, Masaharu Morimoto, Takayuki Kuroda, Dimitris Metaxas, Haifeng Chen

llms multimodal benchmarks

arXiv AI

Jul 10

Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench

arXiv:2607. 08284v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them.

By Siddhartha Jain, Ameya Velingker

llms nlp benchmarks

arXiv AI

Jun 3

Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization

arXiv:2606. 03022v1 Announce Type: cross Abstract: Hallucination in Large Language Models (LLMs), characterized by the generation of content inconsistent with contextual facts or logical constraints -- remains a persistent challenge for reliable deployment.

By Mingkuan Zhao, Wentao Hu, Tianchen Huang, Yuheng Min, Suquan Chen, Yide Gao, Yanbo Zhai, Shuangyong Song, Xuelong Li

llms benchmarks safety

arXiv AI

Jun 10

Non-Parametric Structural Priors for Geometry Theorem Prediction

arXiv:2603. 04852v2 Announce Type: replace Abstract: Multi-step theorem prediction is a central challenge in geometry problem solving.

By Junbo Zhao, Ting Zhang, Can Li, Wei He, Jingdong Wang, Hua Huang

llms rag benchmarks

arXiv AI

Aug 11

Counterfactual Benchmarking and Training for Factuality Consistency and Order-Robust Grounded Reasoning in LLMs over Heterogeneous Knowledge

arXiv:2608. 07838v1 Announce Type: new Abstract: Large language models (LLMs) have increasingly supported response generation grounded in user-provided knowledge spanning heterogeneous structures.

By Shibo Chu, Yuze Liu, Tiehua Zhang, Zhishu Shen, Lianghua He, Haofen Wang, Zhijun Ding

llms fine-tuning benchmarks safety

arXiv AI

Sep 16

Thinking Deeper, Not Longer: Memory-Efficient Test-Time Reasoning with Depth-Recurrent Transformers for Compositional Generalization

arXiv:2603.21676v2 Announce Type: replace-cross Abstract: Standard Transformers have a fixed computational depth, limiting their ability to generalize to tasks that require variable-depth reasoning....

By Hung-Hsuan Chen

llms safety

arXiv AI

Sep 2

ISO-RAG: Isoperimetric Noise Control for Retrieval-Augmented Generation

arXiv:2609.00513v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) mitigates large language models (LLMs) hallucinations, yet conventional dense retrieval struggles with the complex...

By Siyuan Zhang, Hanchen Wang, Dong Wen, Ying Zhang, Wenjie Zhang

llms rag diffusion nlp benchmarks safety

arXiv AI

Jul 1

Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression

arXiv:2601. 08187v3 Announce Type: replace Abstract: Large language models (LLMs) have demonstrated promising capabilities in Text-Attributed Graph (TAG) understanding.

By Zijun Di, Bin Lu, Huquan Kang, Luoyi Fu, Jiaxin Ding, Xiaoying Gan, Lei Zhou, Xinbing Wang

llms benchmarks

Related stories