arXiv AI By Jingjing Wei, Hao Zhao, Xinke Li, Linxi Yu, Xiaosong Gao, Shanshan Xiao

MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure Generation

Read the original on arXiv AI →

MathVis is a new benchmark that tests large language models (LLMs) on their ability to generate and reason with mathematical figures. It contains 512 tasks, each with a mathematical statement, a reference diagram, executable TikZ or SVG code, a gold intermediate representation, atomic rubrics, and error codes. Initial results show that GPT‑6‑Astra, Claude Fable 5, and Gemini 3.1 Pro Preview achieve pass@1 scores of 31.25 %, 26.56 %, and 22.85 % respectively, yet the models tend to produce outputs dominated by formal languages and graphics programs rather than clear, diagram‑mediated reasoning, often resulting in visually confusing solutions for humans.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jun 4

VAMPS: Visual-Assisted Mathematical Problem Solving Benchmark

arXiv:2606. 04244v1 Announce Type: new Abstract: Multimodal large language models are increasingly capable of complex reasoning, yet their performance often degrades when they must externalize a problem through a tool and then reason over the tool's output, specifically when they rely on visual aids.

By Amirhossein Dabiriaghdam, Shayan Vassef, Mohammadreza Bakhtiari, Yasamin Medghalchi, Ilker Hacihaliloglu, Mesrob Ohannessian, Lele Wang, Giuseppe Carenini
arXiv Computation and Language
Sep 10

From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning

The paper introduces a framework that combines a Geometric Vision Parser and a Symbolic Solver to enable a Large Language Model to solve complex plane geometry problems. By translating diagrams into symbolic representations and performing formal deductions, the approach reduces hallucinations and produces interpretable, human-like solutions. Experiments on a new benchmark from 2025 Chinese Zhongkao exams show performance comparable to Gemini 2.5 Pro.

By Weichen Dai, Rafael Medeiros Cabral, Ziyi Shou, Yan Cao, Xin Shen, Dongcai Lu, Yi Zhou
arXiv AI
Aug 20

Solving Is Not Drawing: A Benchmark for Diagrammatic Reasoning in Olympiad Geometry

The paper introduces a new benchmark for diagrammatic reasoning in olympiad geometry, comprising 954 self‑contained problems and a 297‑problem hard subset. Each problem is paired with a human‑authored, high‑fidelity diagram in Asymptote code and a suite of metrics for evaluating diagram construction. Experiments show that current foundation models excel at solving the problems but produce markedly less faithful diagrams, with an average compile success rate of only 36.14%.

By Hsien Xin Peng, Anthony Kim, Alvin Li, Calvin Supasanya, Shivank Garg, Kevin Zhu
arXiv AI
Jun 11

MentisOculi: Revealing the Limits of Reasoning with Mental Imagery

arXiv:2602. 02465v2 Announce Type: replace Abstract: Frontier models are transitioning from multimodal large language models (MLLMs) that merely ingest visual information to unified multimodal models (UMMs) capable of native interleaved generation.

By Jana Zeller, Thadd\"aus Wiedemer, Fanfei Li, Thomas Klein, Prasanna Mayilvahanan, Matthias Bethge, Felix Wichmann, Ryan Cotterell, Wieland Brendel