arXiv AI

RamseyGadgets: A Graph Construction Dataset for LLMs

arXiv:2608. 14999v1 Announce Type: cross Abstract: Constructing special graphs is an important task within graph theory and computer science.

arXiv Machine Learning

6d ago

AutoGrable: What Is a Good Graph for a Table?

arXiv:2608. 11431v1 Announce Type: new Abstract: Graph learning presupposes a graph, and tables and relational databases do not come with one.

By Tamara Cucumides, Floris Geerts

safety

arXiv AI

5d ago

Unified Multi-Dimensional Benchmark for Complex Graph Reasoning in Large Language Models

arXiv:2608. 12391v1 Announce Type: cross Abstract: Graph reasoning provides a promising testbed for evaluating the reasoning ability of large language models (LLMs), as graph instances can be programmatically generated, structurally controlled, and naturally scaled to long-input settings.

By Fali Wang, Ali Al-Lawati, Iliyas Bektas, Jinxuan Fang, Alek Melenski, Tianxiang Zhao, Yao Ma, Suhang Wang

llms rag fine-tuning benchmarks

arXiv AI

Jun 3

GTBench: A Curriculum-Grounded Benchmark for Evaluating LLMs as Mathematical Research Assistants in Graph Theory

arXiv:2606. 03144v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as self-study assistants in technical disciplines, yet their reliability as mathematical reasoning assistants remains poorly understood.

By Noujoud Nader, Ibrahem Aljabea, Patrick Diehl, Deepti Gupta

llms benchmarks

arXiv AI

Aug 11

Neurosymbolic Discovery of Algebraic Graph Constructions

arXiv:2608. 08118v1 Announce Type: new Abstract: There are several methods for searching for graphs with prescribed properties, such as SAT solvers and specialized generators.

By David Seka, Stefan Szeider

llms agents fine-tuning benchmarks

arXiv Machine Learning

Jun 11

GraphInfer-Bench: Benchmarking LLM's Inference Capability on Graphs

arXiv:2606. 11562v1 Announce Type: new Abstract: Graph analysis underlies many applications whose answers cannot be looked up in a single record or retrieved along a path: laundering rings, drug repurposing, user preference, and scientific theme are all inferred from a node together with its neighbourhood.

By Zhuoyi Peng, Jingzhou Jiang, Hanlin Gu, Lixin Fan, Yi Yang

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jun 2

KG-Guard: Graph-Based Hallucination Detection for Knowledge Base Question Answering

arXiv:2606. 00328v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for knowledge base question answering (KBQA), where answering requires selecting entities from a question-specific knowledge-graph subgraph.

By Albert Sawczyn, Piotr Bielak, Tomasz Kajdanowicz

llms rag nlp benchmarks safety

arXiv Machine Learning

Jun 25

Project Auto-World: Towards Automated Benchmarking of Neural Relational Reasoners

arXiv:2606. 24965v1 Announce Type: cross Abstract: Reasoning about relational structures remains a significant challenge for neural models, particularly when they must systematically apply learned knowledge to problem instances that are harder than those seen in training.

By Anirban Das, Joanne Boisson, Irtaza Khalid, Sumita Garai, Steven Schockaert

llms agents benchmarks

arXiv AI

Jun 16

VeriGraph: Towards Verifiable Data-Analytic Agents

arXiv:2606. 16603v1 Announce Type: cross Abstract: LLM-based agents have demonstrated strong capabilities in data-intensive analytical tasks, yet their outputs are rarely verifiable: a reliance on linear text trajectories makes their reasoning difficult to audit.

By Jiajie Jin, Zhao Yang, Wenle Liao, Yuyang Hu, Guanting Dong, Xiaoxi Li, Yutao Zhu, Zhicheng Dou

llms agents benchmarks

arXiv AI

Jul 28

Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers

arXiv:2509. 03059v2 Announce Type: replace-cross Abstract: Recent advances in Large Language Models (LLMs) have shown that their reasoning capabilities can be significantly improved through Reinforcement Learning with Verifiable Reward (RLVR), particularly in domains like mathematics and programming, where ground-truth correctness can be automatically evaluated.

By Xingyue Huang, Rishabh, Gregor Franke, Ziyi Yang, Jiamu Bai, Weijie Bai, Jinhe Bi, Zifeng Ding, Yiqun Duan, Chengyu Fan, Wendong Fan, Xin Gao, Ruohao Guo, Yuan He, Zhuangzhuang He, Xianglong Hu, Neil Johnson, Bowen Li, Fangru Lin, Siyu Lin, Tong Liu, Yunpu Ma, Hao Shen, Hao Sun, Beibei Wang, Fangyijie Wang, Hao Wang, Haoran Wang, Yang Wang, Yifeng Wang, Zhaowei Wang, Ziyang Wang, Yifan Wu, Zikai Xiao, Chengxing Xie, Fan Yang, Junxiao Yang, Qianshuo Ye, Ziyu Ye, Guangtao Zeng, Yuwen Ebony Zhang, Zeyu Zhang, Zihao Zhu, Bernard Ghanem, Philip Torr, Guohao Li

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 2

Knowledge Graphs as the Missing Data Layer for LLM-Based Industrial Asset Operations

arXiv:2605. 26874v2 Announce Type: replace-cross Abstract: LLM-based agents for industrial asset operations show limited accuracy when reasoning over flat document stores.

By Madhulatha Mandarapu, Sandeep Kunkunuru

llms agents benchmarks

arXiv AI

Jul 28

Evolving from Lessons: Skill-Augmented Table Graph Reasoning for Operation-wise Table Question Answering

arXiv:2607. 22633v1 Announce Type: new Abstract: Table Question Answering (TableQA) aims to reason over tables to answer user queries.

By Guixin Su, Qiankun Pi, Mayi Xu, Wenli Li, Ming Zhong, Yuanyuan Zhu, Jiawei Jiang, Tieyun Qian

llms nlp efficiency

arXiv AI

Jul 15

Efficiently Learning Branching Networks for Multitask Algorithmic Reasoning

arXiv:2512. 01113v2 Announce Type: replace-cross Abstract: Algorithmic reasoning -- the ability to perform step-by-step logical inference -- is a synthetic benchmark for evaluating multi-step reasoning abilities, designed for graph neural networks and also for transformer models.

By Dongyue Li, Zhenshuo Zhang, Minxuan Duan, Edgar Dobriban, Hongyang R. Zhang

llms benchmarks

Related stories