The paper introduces OSVerify, an algorithmic framework for one-sided verification that prioritizes rare valid solutions while filtering out invalid candidates to reduce human review workload. OSVerify separates error proposal—searching for errors across solution granularities via hierarchical decomposition—from error validation, which distinguishes major errors from minor or hallucinated ones. Experiments on competition- and research-level benchmarks show a 46% reduction in human reviews of invalid solutions compared to voting baselines, increasing to 57% after post-training the validator. The framework is applied to identify errors in published research papers and to resolve three Langlands open conjectures, confirming its utility in human‑AI research collaboration.
arXiv:2610.18089v1 Announce Type: new Abstract: AI has become increasingly helpful in scientific discovery, yet human expert review remains essential for those problems without reliable automated eva...
arXiv:2606. 28277v1 Announce Type: cross Abstract: Artificial intelligence is driving a revolution in scientific discovery, accelerating everything from hypothesis generation to mathematical theorem proving.
The paper introduces HorizonMath, a benchmark of 113 largely unsolved mathematical problems across eight domains, paired with an open-source framework for automated verification. It focuses on the generator‑verifier gap, targeting problems that are hard to discover but easy to verify computationally, thereby avoiding costly formal proof verification or manual review. Using this framework, the authors found six novel solutions—three each from GPT‑5.4 Pro and GPT‑5.6 Sol—demonstrating that current models can contribute to mathematical research, while most state‑of‑the‑art models score below 10%.
arXiv:2608. 05179v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly used across the scientific research lifecycle: ideation, literature search, experiment design and execution, analysis, manuscript drafting, and review.
arXiv:2607. 28631v1 Announce Type: new Abstract: AI Scientist systems capable of autonomous research have the potential to significantly accelerate scientific discovery.
The paper "Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers" introduces SciSlopBench, a dataset of 390 AI‑generated papers paired with human‑written counterparts, and defines six measures across Structure, Argument, and Artifacts to detect scientific slop. The authors show that these measures can identify AI papers with 85.9% accuracy and that higher slop correlates with lower ICLR ratings and distinguishes rejected from accepted papers. They also propose SciSlopHarness, a framework that guides a fixed LLM to revise only evidence‑supported sections, reducing the AI‑human gap by 63% without human reference targets.
arXiv:2606. 11337v1 Announce Type: new Abstract: Scientific AI agents increasingly retrieve evidence, reason across sources, and synthesize conclusions used in consequential decisions.
The paper introduces ScientistTwo, a fully autonomous multi‑agent framework that takes a scientific problem, establishes baselines, generates hypotheses, and coordinates specialized agents to conduct an end‑to‑end discovery cycle without human intervention. It rigorously tests and refines its methods through automated experiments, ablation studies, and a closed‑loop peer‑review engine. Benchmarking against top conferences (ICLR, ICML, NeurIPS) shows that ScientistTwo produces expert‑level, publishable papers and codebases that outperform human state‑of‑the‑art models and receive higher review ratings under automated AI review.
arXiv:2607. 26064v1 Announce Type: cross Abstract: AI systems are becoming autonomous research agents that generate hypotheses, design experiments, and produce discoveries at scales beyond human oversight.
PAPERBENCHX is a new benchmark that tests whether AI can reproduce scientific experiments across 12 disciplines, including electromagnetics, photonics, chemistry, materials science, biology, and robotics. It consists of 93 expert-constructed tasks, each providing a source paper, a public specification, and a domain-native scientific environment. The benchmark evaluates AI agents by having them reconstruct, execute, and diagnose key experiments, then replay submissions from scratch to score regenerated evidence against expert-validated criteria, revealing that current AI configurations fully reproduce only 14% of tasks.
The paper introduces a verification‑centric framework and benchmark for evaluating large language models (LLMs) as assistants in peer review, focusing on error detection rather than imitation of human reviews. It presents a scalable benchmark that injects synthetic errors into conference papers to test LLMs’ ability to spot logical contradictions, and proposes a Multi‑Layered Review (MLR) framework that emphasizes deep manuscript comprehension before generating reviews. Experiments show the approach aligns well with human review scores, excels at error detection, and offers complementary reviewer insights, while also revealing vulnerabilities to adversarial manipulation.
arXiv:2607. 27191v1 Announce Type: cross Abstract: Forecasts of explosive AI progress hinge on AI agents automating AI research.