← Back to all news
arXiv AI

Beyond Verdicts: A Graph-Based Analysis of Human and LLM Reasoning in Scientific Fact-Checking

The paper introduces a typed reasoning graph framework to compare human and large language model (LLM) reasoning paths in scientific fact‑checking. By modeling explanations as graphs linking false claims to study context, findings, premises, and fallacy labels, the authors enable one‑to‑one alignment of human and LLM reasoning at the sub‑graph level. Using 84 false claims from MISSCIPLUS, they evaluate GPT‑5, Claude Opus 4.7, and Qwen3‑32B, finding distinct performance patterns: Qwen3‑32B has the lowest verdict failure rate, GPT‑5 shows the highest human alignment, and Claude Opus 4.7, while weak at verdict prediction, often produces valid reasoning in successful cases.

Related stories

arXiv Computation and Language
Sep 3

HyGRAIL: Cost-Aware and Evidence-Grounded Scientific Hypothesis Discovery over Knowledge Graphs

HyGRAIL is a framework for discovering scientific hypotheses in incomplete knowledge graphs by combining a graph neural network (GNN) triage with large language model (LLM) review. The GNN scores candidate hypotheses and routes only ambiguous cases to the LLM, which receives structured evidence from the graph converted into natural language. Experiments on MatKG show HyGRAIL achieves the highest F1 score, improves over baselines, and cuts LLM calls by over 54%.

By Yihang Sun, Zhihan Zhu, Zhiyuan Jiang, Jingyi Ge, Zixuan Li, Jiaxuan You
llmsagentsbenchmarks
More like this →
arXiv Machine Learning
Oct 7

Cite What You Explore: Budget-Aware LLM Reasoning over Medical KGs with Verifiable Evidence

The paper introduces BAR, a Budget‑Aware LLM Reasoning framework that enhances post‑discharge risk prediction by integrating external medical knowledge graphs (KGs). BAR refines KGs into disease‑specific evidence graphs with support scores and provenance, then uses an LLM to plan, navigate, and verify evidence within a patient‑specific budget. Experiments on MIMIC‑III and MIMIC‑IV across eight diseases show BAR improves AUPRC by 3.4 points, raises citation precision from 59.8% to 77.9%, and uses only 62‑65% of the allotted budget.

By Chen Chen, Dongjie Wang, Mei Liu, Zijun Yao
llms
More like this →
arXiv Computation and Language
4d ago

On the LLM Verification for Argument Graph Generation

The paper proposes a layered verification approach for argument graphs generated by large language models (LLMs). It suggests starting with deterministic checks, then using solver-based methods, followed by LLM reviews, and resorting to debate only when uncertainty remains, reserving human judgment for cases no other instrument can resolve. A study on human-annotated graphs with injected defects shows that while LLM reviewers can detect structural issues with high recall, they also produce false positives, exhibit inconsistent verdicts, and are slower than exact checks, underscoring the need for deterministic verification gates.

By Jakub B\k{a}ba, Jaros{\l}aw A. Chudziak
llmsagents
More like this →
arXiv AI
Jun 2

Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution

arXiv:2603. 05308v3 Announce Type: replace-cross Abstract: Assessing whether an article supports an assertion is essential for hallucination detection and claim verification.

By Qiao Jin, Yin Fang, Lauren He, Yifan Yang, Guangzhi Xiong, Zhizheng Wang, Nicholas Wan, Joey Chan, Donald C. Comeau, Robert Leaman, Charalampos S. Floudas, Aidong Zhang, Michael F. Chiang, Yifan Peng, Zhiyong Lu
llmsnlpbenchmarkssafety
More like this →
arXiv AI
Jun 16

VeriGraph: Towards Verifiable Data-Analytic Agents

arXiv:2606. 16603v1 Announce Type: cross Abstract: LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable: a reliance on linear text trajectories makes their reasoning difficult to audit.

By Jiajie Jin, Zhao Yang, Wenle Liao, Yuyang Hu, Guanting Dong, Xiaoxi Li, Yutao Zhu, Zhicheng Dou
llmsagentsbenchmarks
More like this →