The paper introduces a two-stage diagnostic evaluation to assess the structural reliability of reasoning graphs used in large language models for tasks like multi-hop question answering. Stage 1 examines how graph reliability depends on topology, prompt design, and model family, while Stage 2 evaluates the impact of graph errors on uncertainty estimation. Findings show that graph-based computation is vulnerable to upstream errors, offers limited benefit as prompt context alone, but improves confidence-based answer discrimination and enables node-level diagnostics.
arXiv:2605. 19228v2 Announce Type: replace-cross Abstract: Large Language Models have achieved strong performance on reasoning tasks with objective answers by generating step-by-step solutions, but diagnosing where a multi-step reasoning trace might fail remains difficult.
arXiv:2606. 03883v1 Announce Type: new Abstract: Large reasoning models (LRMs) are often evaluated using metrics such as final-answer accuracy or token count.
Graph-augmented large language models often assume that graph evidence produced by external computation and placed in the input can be used by the native decoder. We test this assumption with HopQA, a...
The paper introduces GUT, a method that uses directed acyclic graphs to represent all possible reasoning branches of Large Language Models (LLMs). It comprises two modules: GUT-Q, which quantifies reasoning uncertainty by approximating graph complexity, and GUT-O, which reduces uncertainty through reinforcement learning that rewards lower uncertainty. Experiments on four LLMs across five datasets demonstrate GUT’s effectiveness in measuring and mitigating reasoning uncertainty.
arXiv:2603.16654v3 Announce Type: replace-cross Abstract: Evaluating the reasoning abilities of large language models (LLMs) solely from final answers can obscure failures in intermediate steps, espe...
arXiv:2608.30437v1 Announce Type: new Abstract: Graph-augmented large language models often assume that graph evidence produced by external computation and placed in the input can be used by the nati...
arXiv:2606. 03144v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as self-study assistants in technical disciplines, yet their reliability as mathematical reasoning assistants remains poorly understood.
arXiv:2604. 12503v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have shown remarkable capabilities across various tasks but remain prone to hallucinations in knowledge-intensive scenarios.
arXiv:2610.15107v1 Announce Type: new Abstract: Chain-of-thought prompting enables reasoning in large language models but produces explanations that are difficult to verify, expensive to generate, an...
arXiv:2607. 17266v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in natural language processing.
arXiv:2604. 01993v2 Announce Type: replace-cross Abstract: Multi-hop QA benchmarks often reward Large Language Models (LLMs) for spurious correctness, where models reach correct answers through invalid intermediate reasoning.
The paper proposes a layered verification approach for argument graphs generated by large language models (LLMs). It suggests starting with deterministic checks, then using solver-based methods, followed by LLM reviews, and resorting to debate only when uncertainty remains, reserving human judgment for cases no other instrument can resolve. A study on human-annotated graphs with injected defects shows that while LLM reviewers can detect structural issues with high recall, they also produce false positives, exhibit inconsistent verdicts, and are slower than exact checks, underscoring the need for deterministic verification gates.