arXiv AI
Kinship Data Benchmark for Multi-hop Reasoning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 15
Theseus in the Graph: Towards Traceable Multi-Hop Graph Navigation
arXiv:2609.14528v1 Announce Type: cross Abstract: Multi-Hop Knowledge Graph Question Answering (KGQA) tasks require models to assemble relational evidence along paths in a KG to answer natural-langua...
arXiv AI
Jul 17
Enhancing Small Language Models Reasoning through Knowledge Graph Grounding
arXiv:2607. 14149v1 Announce Type: new Abstract: Although large language models (LLMs) have set benchmarks for zero-shot reasoning, their deployment remains cost-prohibitive and environmentally taxing.
arXiv AI
Sep 3
DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents
DocHop is a new benchmark that tests multimodal large language models on integrated chart‑context reasoning within document‑style images. The benchmark presents narrative text that imposes multi‑step compositional constraints, while charts supply the data needed to answer questions grounded in semantic reference labels. It contains 2,074 examples across six task categories, generated via a stochastic logic‑first pipeline that controls reasoning depth and visual density, and shows a large performance gap between humans (over 90% accuracy) and the best models (62.83%).
arXiv AI
Aug 11
Counterfactual Benchmarking and Training for Factuality Consistency and Order-Robust Grounded Reasoning in LLMs over Heterogeneous Knowledge
arXiv:2608. 07838v1 Announce Type: new Abstract: Large language models (LLMs) have increasingly supported response generation grounded in user-provided knowledge spanning heterogeneous structures.
Hugging Face Trending Papers
Jun 8
Multi-Hop Knowledge Composition is Bound by Pretraining Exposure
Large Language Models fail at implicit multi-hop reasoning: a model answers "When was $X$ born? " and "Who is $Y$'s closest friend?
Hugging Face Trending Papers
Jul 9
Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench
Large language models (LLMs) have demonstrated rapidly improving long-context capabilities, prompting a wave of benchmarks designed to evaluate them. However, existing long-context evaluations - from Needle-in-a-Haystack (NIAH) tests to more recent multi-hop reasoning and summarization tasks - predominantly measure average-case performance, and many are either saturated or lack robustness.