arXiv AI

A Human Audit of OpenAIs AI-Generated Mathematical Proofs

The article audits 18 chapter‑specific reviews of ten mathematical results announced by OpenAI, examining review standards, Lean formalizations, subsequent research, and references. No confirmed substantive errors remain in the principal results, though review depth varies and some dependencies are only partially checked. Chapter 8 raises the strongest reservation, requiring major revision of compressed analytic arguments, while Chapter 6’s polarity error was corrected after a PDF extraction issue.

arXiv AI
Aug 26

Auditing an AI-Generated Mathematical Proof: Human Assessment of OpenAI's Quantum Parallel-Repetition Argument

The article reports an independent human audit of a proof from OpenAI’s "Ten Advances in Mathematics and Theoretical Computer Science." An initial review flagged a polarity error in a greedy conditioning lemma, but this was traced to a PDF text‑extraction issue that removed an overbar from a symbol. After correcting the rendering, no mathematical error was found, and the audit underscores that errors in AI‑generated mathematics can stem from document‑processing problems as well as reasoning mistakes.

By Miko{\l}aj Sienicki, Krzysztof Sienicki
arXiv AI
6d ago

Beyond Selected Breakthroughs: Evaluating Language Models on Randomly Sampled Mathematical Conjectures

The paper introduces OpenConjecture, a dataset of mathematical conjectures from arXiv papers since February 2026, and evaluates frontier GPT models on 200 randomly sampled conjectures. The models generate candidate solutions for about 60% of the conjectures, with counterexamples occurring roughly 1.3 times more often than proofs. The study examines solution novelty, citation practices, and fidelity to the original statements, finding that some solutions deviate from authors’ intent and that many proofs closely follow existing results, though some exhibit moderate novelty.

By Helen Jenne, Davis Brown, Jesse He, Henry Kvinge
Hugging Face Trending Papers
Oct 29

Ramanujan Challenge Results: AI-Assisted Evaluation Pipeline for Multi-Format Proofs

The Ramanujan Challenge benchmark tests AI on research‑level number theory problems, including unpublished results and open conjectures. Analysis of submissions shows that AI models, CAS, and formal tools can produce correct solutions that differ conceptually from official ones, even proving a published open conjecture. To reliably judge these complex proofs at scale, the authors propose an AI‑assisted evaluation pipeline that reduces false positives in natural language proofs from 23% to 6% and improves code judgment accuracy from 46% to 58%.

arXiv AI
6d ago

Ramanujan Challenge Results: AI-Assisted Evaluation Pipeline for Multi-Format Proofs

The paper reports on the Ramanujan Challenge, a benchmark of research‑level number theory problems, and evaluates submissions from AI models, CAS, and formal tools. It highlights that some correct solutions differ conceptually from official ones and even proves a published open conjecture. The authors introduce an AI‑assisted evaluation pipeline that significantly lowers false positives in natural language proofs and improves accuracy for supporting code, offering a scalable method for assessing high‑level mathematical work.

By Alan Sorani, Ashvni Narayanan, Michael Shalyt, Tali Monderer, Rotem Kalisch, Hila Barkan, Elyasheev Leibtag, Shachar Weinbaum, Carsten Schneider, John Campbell, Ido Kaminer
arXiv Machine Learning
Sep 11

Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification

The paper introduces HorizonMath, a benchmark of 113 largely unsolved mathematical problems across eight domains, paired with an open-source framework for automated verification. It focuses on the generator‑verifier gap, targeting problems that are hard to discover but easy to verify computationally, thereby avoiding costly formal proof verification or manual review. Using this framework, the authors found six novel solutions—three each from GPT‑5.4 Pro and GPT‑5.6 Sol—demonstrating that current models can contribute to mathematical research, while most state‑of‑the‑art models score below 10%.

By Erik Y. Wang, Sumeet R. Motwani, James V. Roggeveen, Eliot Hodges, Dulhan Jayalath, Charles London, Kalyan Ramakrishnan, Jakob Foerster, Cheng Zhang, Flaviu Cipcigan, Philip Torr, Alessandro Abate