arXiv AI
Omanic: Towards Step-wise Evaluation of Multi-hop Reasoning in Large Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 10
SAFE: An LLM-as-Verifier Framework for Evidence-Grounded Multi-Hop Reasoning
arXiv:2604. 01993v2 Announce Type: replace-cross Abstract: Multi-hop QA benchmarks often reward Large Language Models (LLMs) for spurious correctness, where models reach correct answers through invalid intermediate reasoning.
arXiv AI
Sep 3
PRO-Step: Step-level Process Reward Optimization for Retrieval-Augmented Generation
PRO-Step introduces a step‑level process reward optimization framework for Retrieval‑Augmented Generation (RAG) that evaluates both logical validity and evidential grounding at each reasoning step. By training a generative Preference‑Based Reward Model (PRM) and using PRM‑guided value tree search to create preference pairs, the method optimizes the policy through step‑level Direct Preference Optimization. Experiments on single and multi‑hop QA benchmarks show that PRO‑STEP achieves the best average EM and F1 scores across five datasets.
arXiv Computation and Language
Aug 25
Beyond Factual Knowledge: Benchmarking and Learning Step-Level Procedural Rule Reasoning in Large Language Models
arXiv:2608.22753v1 Announce Type: new Abstract: Large language models (LLMs) excel at text understanding and generation, yet still struggle to reliably understand and apply externally provided proced...
arXiv AI
Jul 28
Reason Popper-ly: Patching In-Context Reasoning with Inductive Logic Programming
arXiv:2607. 23019v1 Announce Type: new Abstract: Chain-of-thought (CoT) prompting enables large language models (LLMs) to tackle multi-step reasoning tasks, yet the generated intermediate steps are not guaranteed to be logically sound.
arXiv AI
Jul 14
CRiT-QA: Evaluating Multi-hop Reasoning with Counterfactual Chains and Distractor Traps
arXiv:2607. 10562v1 Announce Type: new Abstract: Evaluating the multi-hop reasoning capabilities of large language models remains a significant challenge.
arXiv AI
Jun 6
ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces
arXiv:2606. 05402v1 Announce Type: cross Abstract: Large reasoning models (LRMs) produce reasoning traces with non-linear structures, such as backtracking and self-correction, that complicate the evaluation and monitoring of the reasoning process.