arXiv:2608. 14999v1 Announce Type: cross Abstract: Constructing special graphs is an important task within graph theory and computer science.
By Zohair Raza Hassan, Deepak Pandita
MathVis is a new benchmark that tests large language models (LLMs) on their ability to generate and reason with mathematical figures. It contains 512 tasks, each with a mathematical statement, a reference diagram, executable TikZ or SVG code, a gold intermediate representation, atomic rubrics, and error codes. Initial results show that GPT‑6‑Astra, Claude Fable 5, and Gemini 3.1 Pro Preview achieve pass@1 scores of 31.25 %, 26.56 %, and 22.85 % respectively, yet the models tend to produce outputs dominated by formal languages and graphics programs rather than clear, diagram‑mediated reasoning, often resulting in visually confusing solutions for humans.
By Jingjing Wei, Hao Zhao, Xinke Li, Linxi Yu, Xiaosong Gao, Shanshan Xiao
arXiv:2602. 16902v4 Announce Type: replace Abstract: We introduce LLM-Wikirace, a benchmark for evaluating planning, reasoning, and world knowledge in large language models (LLMs).
By Juliusz Ziomek, William Bankes, Lorenz Wolf, Shyam Sundhar Ramesh, Xiaohang Tang, Ilija Bogunovic
arXiv:2511. 10234v3 Announce Type: replace-cross Abstract: While promising, graph reasoners based on Large Language Models (LLMs) lack built-in invariance to symmetries in graph representations.
By Daniel Herbst, Lea Karbevska, Divyanshu Kumar, Akanksha Ahuja, Fatemeh Gholamzadeh Nasrabadi, Fabrizio Frasca
The paper introduces a new benchmark for diagrammatic reasoning in olympiad geometry, comprising 954 self‑contained problems and a 297‑problem hard subset. Each problem is paired with a human‑authored, high‑fidelity diagram in Asymptote code and a suite of metrics for evaluating diagram construction. Experiments show that current foundation models excel at solving the problems but produce markedly less faithful diagrams, with an average compile success rate of only 36.14%.
By Hsien Xin Peng, Anthony Kim, Alvin Li, Calvin Supasanya, Shivank Garg, Kevin Zhu
arXiv:2602.00276v3 Announce Type: replace
Abstract: Large language models (LLMs) have demonstrated strong reasoning capabilities on math and coding, but frequently fail on symbolic classical planning...
By Aditya Kumar, William W. Cohen