arXiv AI
One-Sided Verification Under Limited Expert Review
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Oct 13
One-Sided Verification Under Limited Expert Review
The paper introduces OSVerify, an algorithmic framework for one-sided verification that prioritizes rare valid solutions while filtering out invalid candidates to reduce human review workload. OSVerify separates error proposal—searching for errors across solution granularities via hierarchical decomposition—from error validation, which distinguishes major errors from minor or hallucinated ones. Experiments on competition- and research-level benchmarks show a 46% reduction in human reviews of invalid solutions compared to voting baselines, increasing to 57% after post-training the validator. The framework is applied to identify errors in published research papers and to resolve three Langlands open conjectures, confirming its utility in human‑AI research collaboration.
arXiv AI
Jun 29
Towards Automating Scientific Review with Google's Paper Assistant Tool
arXiv:2606. 28277v1 Announce Type: cross Abstract: Artificial intelligence is driving a revolution in scientific discovery, accelerating everything from hypothesis generation to mathematical theorem proving.
arXiv Machine Learning
Sep 11
Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification
The paper introduces HorizonMath, a benchmark of 113 largely unsolved mathematical problems across eight domains, paired with an open-source framework for automated verification. It focuses on the generator‑verifier gap, targeting problems that are hard to discover but easy to verify computationally, thereby avoiding costly formal proof verification or manual review. Using this framework, the authors found six novel solutions—three each from GPT‑5.4 Pro and GPT‑5.6 Sol—demonstrating that current models can contribute to mathematical research, while most state‑of‑the‑art models score below 10%.
arXiv AI
Oct 2
Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers
The paper "Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers" introduces SciSlopBench, a dataset of 390 AI‑generated papers paired with human‑written counterparts, and defines six measures across Structure, Argument, and Artifacts to detect scientific slop. The authors show that these measures can identify AI papers with 85.9% accuracy and that higher slop correlates with lower ICLR ratings and distinguishes rejected from accepted papers. They also propose SciSlopHarness, a framework that guides a fixed LLM to revise only evidence‑supported sections, reducing the AI‑human gap by 63% without human reference targets.
arXiv AI
Aug 3
Can AI Evaluate AI Scientists? A Benchmarking Study of Autonomous Research Generation Systems Using Automated Multi-Model Review
arXiv:2607. 28631v1 Announce Type: new Abstract: AI Scientist systems capable of autonomous research have the potential to significantly accelerate scientific discovery.
arXiv AI
Oct 31
PAPERBENCHX: Can AI Reproduce Science Across Disciplines?
PAPERBENCHX is a new benchmark that tests whether AI can reproduce scientific experiments across 12 disciplines, including electromagnetics, photonics, chemistry, materials science, biology, and robotics. It consists of 93 expert-constructed tasks, each providing a source paper, a public specification, and a domain-native scientific environment. The benchmark evaluates AI agents by having them reconstruct, execute, and diagnose key experiments, then replay submissions from scratch to score regenerated evidence against expert-validated criteria, revealing that current AI configurations fully reproduce only 14% of tasks.