arXiv AI

GUT: Quantifying and Optimizing the Reasoning Uncertainty of LLMs via Graph Complexity

The paper introduces GUT, a method that uses directed acyclic graphs to represent all possible reasoning branches of Large Language Models (LLMs). It comprises two modules: GUT-Q, which quantifies reasoning uncertainty by approximating graph complexity, and GUT-O, which reduces uncertainty through reinforcement learning that rewards lower uncertainty. Experiments on four LLMs across five datasets demonstrate GUT’s effectiveness in measuring and mitigating reasoning uncertainty.

arXiv AI
Aug 14

Unified Multi-Dimensional Benchmark for Complex Graph Reasoning in Large Language Models

arXiv:2608. 12391v1 Announce Type: cross Abstract: Graph reasoning provides a promising testbed for evaluating the reasoning ability of large language models (LLMs), as graph instances can be programmatically generated, structurally controlled, and naturally scaled to long-input settings.

By Fali Wang, Ali Al-Lawati, Iliyas Bektas, Jinxuan Fang, Alek Melenski, Tianxiang Zhao, Yao Ma, Suhang Wang
arXiv AI
Jun 16

VeriGraph: Towards Verifiable Data-Analytic Agents

arXiv:2606. 16603v1 Announce Type: cross Abstract: LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable: a reliance on linear text trajectories makes their reasoning difficult to audit.

By Jiajie Jin, Zhao Yang, Wenle Liao, Yuyang Hu, Guanting Dong, Xiaoxi Li, Yutao Zhu, Zhicheng Dou
arXiv AI
Jul 10

Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Framework

arXiv:2607. 08017v1 Announce Type: cross Abstract: Large-Language Models (LLMs) can be prone to flawed and unfaithful reasoning that decoding strategies like Self-Consistency (SC) fail to detect as they evaluate only final-answer agreement while ignoring the logical validity of intermediate steps.

By Riccardo Revalor, Jalees Rehman, Debjit Pal