The paper introduces HorizonMath, a benchmark of 113 largely unsolved mathematical problems across eight domains, paired with an open-source framework for automated verification. It focuses on the generator‑verifier gap, targeting problems that are hard to discover but easy to verify computationally, thereby avoiding costly formal proof verification or manual review. Using this framework, the authors found six novel solutions—three each from GPT‑5.4 Pro and GPT‑5.6 Sol—demonstrating that current models can contribute to mathematical research, while most state‑of‑the‑art models score below 10%.
By Erik Y. Wang, Sumeet R. Motwani, James V. Roggeveen, Eliot Hodges, Dulhan Jayalath, Charles London, Kalyan Ramakrishnan, Jakob Foerster, Cheng Zhang, Flaviu Cipcigan, Philip Torr, Alessandro Abate
ReuF2F is a Lean 4 benchmark comprising 244 formalized open problems sourced from Research Experiences for Undergraduates (REU). It aims to evaluate coding agents on less prominent, entry‑level research problems across diverse mathematical subfields, using a Docker‑based Lean project setup and a unified harness called Leanimum‑agent. The study finds that even the best model solves only about 40 % of the problems, indicating significant room for improvement.
By Zeyu Zheng, Yiyang Liu, Yuyang Zhao, Congmin Zheng, Jiachen Zhu, Jianian Chen, Kangning Zhang, Kayue Daniel Wong, Lingyu Yang, Rong Shan, Shihao Chen, Thomas Zhu, Wentao Zhang, Xiaopeng Li, Yan Hu, Yongxi Lin, Yuxiang Chen, Zhuoying Ou, Jianghao Lin, Jeremy Avigad, Prasad Tetali, Jingbang Chen, Sean Welleck
arXiv:2610.09769v1 Announce Type: cross
Abstract: Large language models are increasingly contributing to mathematical research, where progress often depends on efficient proof search, incremental imp...
By Adri\'an Z\'ame\v{c}n\'ik, Mat\v{e}j Kripner, Martin Kouteck\'y, Martin Balko, Jan Greb\'ik, Pavel Hub\'a\v{c}ek, Robert \v{S}\'amal, V\'aclav Rozho\v{n}
The paper presents a system that transforms open‑problem listings in research fields into a structured, queryable knowledge graph. It combines a hand‑curated problem layer with an evidence layer derived from bibliographic data, linked through typed, provenance‑carrying connections. The authors evaluate the approach on database and AI problem catalogs, showing that the new graph can answer all previously answerable queries and additional ones about recent work, but also reveal limitations in automated status judgments, leading to a design recommendation that status changes must be proposed with evidence and reviewed by humans.
By Madhulatha Mandarapu, Sandeep Kunkunuru
Bolzano is a multi‑agent, open‑source system that employs parallel prover agents and a verifier agent to conduct proof search while maintaining a human‑readable research state. In initial manual trials on expert‑selected problems, it produced eight proofs verified by domain experts. When deployed without problem‑specific guidance on roughly 3,800 open problems from four paper sets, Bolzano solved about 200 of them, including four questions from papers accepted to the STOC 2026 conference, which were confirmed by the authors.
By Adri\'an Z\'ame\v{c}n\'ik, Mat\v{e}j Kripner, Martin Kouteck\'y, Martin Balko, Jan Greb\'ik, Pavel Hub\'a\v{c}ek, Robert \v{S}\'amal, V\'aclav Rozho\v{n}
SciRerankBench is a benchmark designed to evaluate rerankers in retrieval‑augmented large language models (RAG‑LLMs) for scientific literature question answering. It covers five scientific subjects and introduces three types of question‑context‑answer pairs—Noisy Contexts, Semantically Similar but Logically Irrelevant Contexts, and Counterfactual Contexts—to test noise resilience, relevance disambiguation, and factual consistency. The study evaluates 13 popular rerankers across five LLM families, offering detailed insights into their strengths and limitations.
By Haotian Chen, Qingqing Long, Meng Xiao, Xiao Luo, Wei Ju, Chengrui Wang, Yuanchun Zhou, Xuezhi Wang, Hengshu Zhu
arXiv:2604. 13201v2 Announce Type: replace-cross Abstract: Large language models are emerging as scientific assistants, but evaluating their ability to reason from empirical data remains challenging.
By Oliver Bentham, Vivek Srikumar
arXiv:2607. 14178v1 Announce Type: new Abstract: Recent advances in Large Language Models have fueled autonomous AI agents capable of tackling complex scientific tasks, yet existing automated research systems remain predominantly focused on empirically driven domains with quantitative benchmarks, leaving theory-driven discovery, particularly in mathematically grounded disciplines requiring rigorous proofs and synthesis of domain knowledge, largely underexplored.
By Yutong He, Daibo Li, Guohong Li, Jiahe Geng, Zhengyang Huang, Can Ren, Zekun Zhang, Yifan Liu, Shuchen Zhu, Hengrui Zhang, Boao Kong, Ming Sun, Shu Li, Chenyi Li, Jiang Hu, Kun Yuan, Zaiwen Wen, Pingwen Zhang
arXiv:2609.16519v1 Announce Type: new
Abstract: Scientific research increasingly relies on large, heterogeneous data sources, motivating interest in retrieval-augmented generation (RAG) systems that...
By Bernie Boscoe, Srinath Saikrishnan, Vikram Seenivasan, Jack Stark, Andrew Lizarraga, Morgan Himes, Jonathan Soriano, PJ Allen, Tuan Do
Benchmark saturation and data contamination increasingly obscure genuine scientific reasoning in frontier LLMs. We introduce \textsc{ScienceArena}, an olympiad-style benchmark from thirteen public sci...
arXiv:2606. 06526v1 Announce Type: new Abstract: Large language models have made substantial progress on mathematical reasoning, but existing benchmarks typically evaluate well-specified problems with final answers, step-by-step solutions, or complete proofs.
By Sherin Muckatira, Jesse Geneson, Slava Gerovitch, Pavel Etingof, Mikhail Gronas, Anna Rumshisky
Scientific research increasingly relies on large, heterogeneous data sources, motivating interest in retrieval-augmented generation (RAG) systems that provide natural language access to scientific kno...