arXiv AI By Shuang Liang, Xin-Yu Hu, Xiang-Jun Ou, Shao-Qun Zhang

GUT: Quantifying and Optimizing the Reasoning Uncertainty of LLMs via Graph Complexity

Read the original on arXiv AI →

The paper introduces GUT, a method that uses directed acyclic graphs to represent all possible reasoning branches of Large Language Models (LLMs). It comprises two modules: GUT-Q, which quantifies reasoning uncertainty by approximating graph complexity, and GUT-O, which reduces uncertainty through reinforcement learning that rewards lower uncertainty. Experiments on four LLMs across five datasets demonstrate GUT’s effectiveness in measuring and mitigating reasoning uncertainty.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Oct 21

Diagnostic Analysis for Large Language Model Reasoning

The paper introduces a two-stage diagnostic evaluation to assess the structural reliability of reasoning graphs used in large language models for tasks like multi-hop question answering. Stage 1 examines how graph reliability depends on topology, prompt design, and model family, while Stage 2 evaluates the impact of graph errors on uncertainty estimation. Findings show that graph-based computation is vulnerable to upstream errors, offers limited benefit as prompt context alone, but improves confidence-based answer discrimination and enables node-level diagnostics.

By Xinyu Liu, Dongqi Fu, Jun Wu