GraphDecide is a model‑independent benchmark designed to evaluate System One models—such as Jev—that make decisions directly from supplied options on graph‑related tasks. The benchmark combines structural task profiles, matched graph‑text input contrasts, and heuristic‑proposal controls to diagnose graph decision performance. In testing fourteen model‑interface configurations, GraphDecide shows that accurate adjacency recognition does not guarantee broader structural correctness, joint graph‑text input does not consistently improve prediction, and feasible construction does not ensure high solution quality.
arXiv:2608. 12391v1 Announce Type: cross Abstract: Graph reasoning provides a promising testbed for evaluating the reasoning ability of large language models (LLMs), as graph instances can be programmatically generated, structurally controlled, and naturally scaled to long-input settings.
By Fali Wang, Ali Al-Lawati, Iliyas Bektas, Jinxuan Fang, Alek Melenski, Tianxiang Zhao, Yao Ma, Suhang Wang
arXiv:2606. 11562v1 Announce Type: new Abstract: Graph analysis underlies many applications whose answers cannot be looked up in a single record or retrieved along a path: laundering rings, drug repurposing, user preference, and scientific theme are all inferred from a node together with its neighbourhood.
By Zhuoyi Peng, Jingzhou Jiang, Hanlin Gu, Lixin Fan, Yi Yang
Graph-augmented large language models often assume that graph evidence produced by external computation and placed in the input can be used by the native decoder. We test this assumption with HopQA, a...
arXiv:2608. 12097v1 Announce Type: new Abstract: Rubric-based evaluators commonly treat rubrics as prompt context or flat criteria: they specify what to judge but leave criterion composition implicit, even when natural-language rules state it.
By Xi Chen, Jie Mu, Mo Xuan, Qun Shao
arXiv:2608.30437v1 Announce Type: new
Abstract: Graph-augmented large language models often assume that graph evidence produced by external computation and placed in the input can be used by the nati...
By Xiaoyu Guo, Pengcheng Chen, Jiong Yu, Yi Lu, Yaohua Wang, Ziyang Li