arXiv AI
Aug 7
An Axiomatic Benchmark for Evaluation of Scientific Novelty Metrics
arXiv:2604. 15145v2 Announce Type: replace Abstract: The rigorous evaluation of the novelty of a scientific paper is, even for human scientists, a challenging task.
Beyond Correctness: Toward Automated Novelty Verification with Lean 4
arXiv:2608. 14669v1 Announce Type: new Abstract: Artificial intelligence systems applied to mathematics verify correctness but not novelty: an automatically generated theorem can compile in Lean without errors and yet be an already known result.
Related stories
arXiv AI
Jun 30
Faults in Our Formal Benchmarking: Dataset Defects and Evaluation Failures in Lean Theorem Proving
arXiv:2606. 29493v1 Announce Type: new Abstract: Benchmarks for LLM-assisted theorem proving in Lean are often treated as intrinsically reliable because every solved instance comes with a machine-checked proof.
arXiv AI
1d ago
Does the Proof Prove It That Way? Faithful Formalization of Elements Proofs
arXiv:2608. 15432v1 Announce Type: new Abstract: In formal verification, both the autoformalization of statements and automated proof search have been studied extensively.
arXiv AI
Jul 1
RARE: Redundancy-Aware Retrieval Evaluation Framework for High-Similarity Corpora
arXiv:2604. 19047v2 Announce Type: replace-cross Abstract: Existing QA benchmarks typically assume distinct documents with minimal overlap, yet real-world retrieval-augmented generation (RAG) systems operate on corpora such as financial reports, legal codes, and patents, where information is highly redundant and documents exhibit strong inter-document similarity.
arXiv AI
Jun 16
SorryDB: Can AI Provers Complete Real-World Lean Theorems?
arXiv:2603. 02668v2 Announce Type: replace Abstract: We present SorryDB, a dynamically-updating benchmark of open Lean tasks drawn from 78 real world formalization projects on GitHub.
arXiv AI
Jun 11
On the Limits of LLM-as-Judge for Scientific Novelty Assessment
arXiv:2606. 12071v1 Announce Type: cross Abstract: LLMs are increasingly used to generate and judge scientific ideas.
arXiv AI
Jun 9
TheoremBench: Evaluating LLMs on Theorem Proving in Formal Mathematics
arXiv:2606. 09450v1 Announce Type: new Abstract: LLMs have recently achieved strong results on formal proving benchmarks.
arXiv AI
Jul 2
Theoria: Rewrite-Acceptability Verification over Informal Reasoning States
arXiv:2607. 01223v1 Announce Type: new Abstract: When should an AI system's answer be trusted?
arXiv AI
Aug 3
LLM Framework for Discovering Major Mathematical Conjectures: AI's Quest for the Next Riemann Hypothesis
arXiv:2607. 28632v1 Announce Type: new Abstract: Major mathematical conjectures still depend heavily on expert intuition, so a unified method for the systematic generation and validation of conjectures with substantial mathematical potential remains unavailable.
Hugging Face Trending Papers
Jun 8
TheoremBench: Evaluating LLMs on Theorem Proving in Formal Mathematics
LLMs have recently achieved strong results on formal proving benchmarks. However, existing evaluations remain heavily concentrated on competition-style problems and often fail to capture how models behave on longer, more dependency-rich mathematical developments.
arXiv AI
Jun 30
Geometric Measurements of the Axiom of Choice in Neural Proof Embeddings
arXiv:2606. 28572v1 Announce Type: cross Abstract: The axiom of choice has divided the foundations of mathematics for over a century, but the distinction between classical and constructive proofs has remained a philosophical and methodological one.
arXiv AI
1d ago
ALPS: Measuring Valid Creativity in Large Language Models with Mathematical Construction
arXiv:2608. 15979v1 Announce Type: new Abstract: Large language models produce outputs presented as discoveries - new proofs, conjectures, or molecules.