arXiv:2607. 09328v2 Announce Type: replace-cross Abstract: Answering complex questions over long documents frequently requires integrating evidence that the source itself disperses naturally across distant passages.
arXiv:2607. 09328v1 Announce Type: cross Abstract: Answering complex questions over long documents frequently requires integrating evidence that the source itself disperses naturally across distant passages.
GraphNIAH is a new benchmark for long‑context reasoning that uses real citation relationships among scientific papers published after a model’s knowledge cutoff. It automatically samples subgraphs of the citation graph and turns each into a long‑context QA example with a verifiable answer, avoiding synthetic data, dataset reuse, manual annotation, and trivial reasoning steps. Evaluation of eight large language models shows that accuracy drops sharply as reasoning depth increases, indicating that current models often fail to fully explore citation relationships even when all necessary facts are present.
arXiv:2609.38406v1 Announce Type: new Abstract: Access to real-world information is often noisy and fragmented. Constructing a coherent narrative from such fragments requires models to reconstruct mi...
NS-ST-GraphRAG is a neuro‑symbolic spatio‑temporal GraphRAG framework designed to process long‑form literary narratives by integrating ontology‑guided extraction, deterministic constraint checking, dual temporal coordinates, spatial scene attributes, and dynamic sub‑graph retrieval. It selects the appropriate graph state based on the temporal and spatial scope of a query, grounding generated answers in traceable evidence. The authors also introduce Red‑Chamber‑QA, an open multi‑hop question‑answering benchmark for classical Chinese literature, and report that NS‑ST‑GraphRAG outperforms a frozen‑window baseline and a closed‑book model on a held‑out 120‑question split.
arXiv:2610.08388v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong capabilities in question answering, yet they still frequently suffer from hallucinations on kno...
The paper introduces ElephantBench, a closed‑book knowledge probe with 1,094 multi‑account factual questions generated via an auditable graph‑based pipeline that pulls documents from a low‑exposure web corpus and identifies naturally occurring disagreements. Across 32 large language models, even the best model only recovers both divergent accounts on 52.4% of questions, and most models recall one account while omitting the other, indicating persistent epistemic myopia. The study shows that scaling model size and inference‑time reasoning improves recall but does not eliminate incompleteness, and that exposure imbalance in the corpus biases models toward the dominant account.
arXiv:2609.37469v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) grounds large language models in external sources, but retrieved passages often name the right entities without...
The paper introduces LURCH, a benchmark of 6,550 question‑answering instances that quantifies long‑context reasoning difficulty using three properties: evidence scope, evidence dispersion, and reasoning depth. Experiments with both proprietary and open‑weight models show that these properties are strongly negatively correlated with performance, that performance drops sharply for dispersed evidence as context length grows, and that difficulty increases when multiple properties are high simultaneously. The study demonstrates that context length alone is insufficient to explain model failures, and that the new metrics provide a more nuanced understanding of long‑context challenges.
arXiv:2610.27597v1 Announce Type: cross Abstract: Existing reasoning evaluations typically provide task-relevant information to models as available premises, rarely examining concurrently how models...
The paper presents a method that uses relation knowledge—either manually crafted or learned from training data—to select a compact set of knowledge‑graph facts for large language models (LLMs). It introduces derivability, consistency, and validation checks to determine whether an answer can be derived from the selected facts, whether the facts conflict, and whether the derived answer should replace the LLM’s output. Experiments show that seven open LLMs perform better with the compact context than with relevance‑ranked facts, achieving higher exact‑match accuracy on ComplexWebQuestions and outperforming SubgraphRAG across 53 LLM, benchmark, and prompt settings.
LLM agents increasingly answer questions over dynamic raw-document collections, where files may change before preprocessing, and relevant evidence (spans, sections, pages, or tables) is query-dependent. Existing retrieval-augmented approaches pre-materialize evidence via fixed chunking, embeddings, or persistent indexes: effective for lookup, yet costly, stale-prone, and committed to a granularity before the query is known.