arXiv Computation and Language
Nov 2

GraphNIAH: A Benchmark for Long-Context Reasoning over Real-World Graphs

GraphNIAH is a new benchmark for long‑context reasoning that uses real citation relationships among scientific papers published after a model’s knowledge cutoff. It automatically samples subgraphs of the citation graph and turns each into a long‑context QA example with a verifiable answer, avoiding synthetic data, dataset reuse, manual annotation, and trivial reasoning steps. Evaluation of eight large language models shows that accuracy drops sharply as reasoning depth increases, indicating that current models often fail to fully explore citation relationships even when all necessary facts are present.

By Minh-Nam Tran, Abulhair Saparov
arXiv Computation and Language
Sep 7

NS-ST-GraphRAG: Neuro-Symbolic Spatio-Temporal GraphRAG for Literary Knowledge Processing

NS-ST-GraphRAG is a neuro‑symbolic spatio‑temporal GraphRAG framework designed to process long‑form literary narratives by integrating ontology‑guided extraction, deterministic constraint checking, dual temporal coordinates, spatial scene attributes, and dynamic sub‑graph retrieval. It selects the appropriate graph state based on the temporal and spatial scope of a query, grounding generated answers in traceable evidence. The authors also introduce Red‑Chamber‑QA, an open multi‑hop question‑answering benchmark for classical Chinese literature, and report that NS‑ST‑GraphRAG outperforms a frozen‑window baseline and a closed‑book model on a held‑out 120‑question split.

By Zheng Kui Lin