The paper introduces OSVerify, an algorithmic framework for one-sided verification that prioritizes rare valid solutions while filtering out invalid candidates to reduce human review workload. OSVerify separates error proposal—searching for errors across solution granularities via hierarchical decomposition—from error validation, which distinguishes major errors from minor or hallucinated ones. Experiments on competition- and research-level benchmarks show a 46% reduction in human reviews of invalid solutions compared to voting baselines, increasing to 57% after post-training the validator. The framework is applied to identify errors in published research papers and to resolve three Langlands open conjectures, confirming its utility in human‑AI research collaboration.
arXiv:2606. 28277v1 Announce Type: cross Abstract: Artificial intelligence is driving a revolution in scientific discovery, accelerating everything from hypothesis generation to mathematical theorem proving.
By Rajesh Jayaram, Drew Tyler, David Woodruff, Corinna Cortes, Yossi Matias, Vahab Mirrokni, Vincent Cohen-Addad
The paper introduces HorizonMath, a benchmark of 113 largely unsolved mathematical problems across eight domains, paired with an open-source framework for automated verification. It focuses on the generator‑verifier gap, targeting problems that are hard to discover but easy to verify computationally, thereby avoiding costly formal proof verification or manual review. Using this framework, the authors found six novel solutions—three each from GPT‑5.4 Pro and GPT‑5.6 Sol—demonstrating that current models can contribute to mathematical research, while most state‑of‑the‑art models score below 10%.
By Erik Y. Wang, Sumeet R. Motwani, James V. Roggeveen, Eliot Hodges, Dulhan Jayalath, Charles London, Kalyan Ramakrishnan, Jakob Foerster, Cheng Zhang, Flaviu Cipcigan, Philip Torr, Alessandro Abate
The paper "Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers" introduces SciSlopBench, a dataset of 390 AI‑generated papers paired with human‑written counterparts, and defines six measures across Structure, Argument, and Artifacts to detect scientific slop. The authors show that these measures can identify AI papers with 85.9% accuracy and that higher slop correlates with lower ICLR ratings and distinguishes rejected from accepted papers. They also propose SciSlopHarness, a framework that guides a fixed LLM to revise only evidence‑supported sections, reducing the AI‑human gap by 63% without human reference targets.
By Yerim Oh, Young-Jun Lee, Jaewoo Ahn, Gunhee Kim, Dongyeop Kang
arXiv:2607. 28631v1 Announce Type: new Abstract: AI Scientist systems capable of autonomous research have the potential to significantly accelerate scientific discovery.
By Vaibhava Lakshmi Ravideshik, Mayank Kejriwal
PAPERBENCHX is a new benchmark that tests whether AI can reproduce scientific experiments across 12 disciplines, including electromagnetics, photonics, chemistry, materials science, biology, and robotics. It consists of 93 expert-constructed tasks, each providing a source paper, a public specification, and a domain-native scientific environment. The benchmark evaluates AI agents by having them reconstruct, execute, and diagnose key experiments, then replay submissions from scratch to score regenerated evidence against expert-validated criteria, revealing that current AI configurations fully reproduce only 14% of tasks.
By Ruoyu Wu, Zengji Tu, Aolong Sun, Tianyi Ma, Junren Li, Xinbo Xu, Li Fu, Ziyan Zhang, Pinrui Huang, Xiaochen Wang, Chengda Wu, Yutao Zhou, Jianjun Wu, Lin Chang, Baobao Chang, Liang Chen, Kuan Li
arXiv:2608. 05179v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly used across the scientific research lifecycle: ideation, literature search, experiment design and execution, analysis, manuscript drafting, and review.
By Tianyu Ding, Aditya Nannapaneni, Bingfan Liu, Ling Zhang
The paper introduces ScientistTwo, a fully autonomous multi‑agent framework that takes a scientific problem, establishes baselines, generates hypotheses, and coordinates specialized agents to conduct an end‑to‑end discovery cycle without human intervention. It rigorously tests and refines its methods through automated experiments, ablation studies, and a closed‑loop peer‑review engine. Benchmarking against top conferences (ICLR, ICML, NeurIPS) shows that ScientistTwo produces expert‑level, publishable papers and codebases that outperform human state‑of‑the‑art models and receive higher review ratings under automated AI review.
By Jaehyun Nam, Jinsung Yoon, Yanzhou Pan, Yubo Wang, Rui Meng, Parthasarathy Ranganathan, Tomas Pfister
The paper introduces a verification‑centric framework and benchmark for evaluating large language models (LLMs) as assistants in peer review, focusing on error detection rather than imitation of human reviews. It presents a scalable benchmark that injects synthetic errors into conference papers to test LLMs’ ability to spot logical contradictions, and proposes a Multi‑Layered Review (MLR) framework that emphasizes deep manuscript comprehension before generating reviews. Experiments show the approach aligns well with human review scores, excels at error detection, and offers complementary reviewer insights, while also revealing vulnerabilities to adversarial manipulation.
By Rachel S. Y. Teo, Yutaro Yamada, Shashank Kotyan, Yuki Imajuku, Tarin Clanuwat
EPOCH is an evidence‑governed architecture for AI research agents that improves how candidate discoveries are evaluated and validated. By combining explicit task contracts, typed memory, active falsification, admission checks, and independent replay, EPOCH ensures that each candidate is judged against the strength and scope of its claim. The system achieves state‑of‑the‑art performance on several benchmarks, including AlgoTune, Math14, and AgentHPO, and demonstrates significant advances in executable constructions, algorithm optimization, counterexamples, and proof‑supported results across ten discovery problems.
By Binjie Guo, Aisheng Mo, Ruitong Li, Xinle Deng
PaperDoctor is an agent framework that provides evidence‑grounded, actionable feedback for scientific papers before submission. It evaluates writing, layout, references, code, theory, prior work, and experiments through a three‑layer hierarchical system, linking each critique to specific evidence and revision suggestions. The system selectively rebuilds and reruns experiments to uncover reproducibility gaps, and an interactive interface lets authors explore findings tied to their manuscript.
By Kevin Qinghong Lin, Siyuan Hu, Pan Lu, Yu Chen, Yanzhe Chen, Owen Queen, Yupeng Chen, Jialin Yu, Junchi Yu, Zifeng Ding, Yuanfeng Ji, Sheng Liu, Jindong Gu, Linjie Li, Mike Zheng Shou, Philip Torr, James Zou
The paper introduces a new human‑AI collaboration paradigm for mathematical discovery, shifting from selecting individual problems to exploring broad research directions. It presents the Find, Attempt, and Recommend (FAR) pipeline, which automatically searches a literature corpus, filters candidate conjectures, and surfaces promising resolutions for expert review. In a combinatorics pilot, FAR processed over 5,000 papers, identified thousands of open conjectures, and ultimately highlighted 77 items that led to new discoveries.
By Zeyu Zheng, Shengtong Zhang, Jeremy Avigad, Prasad Tetali, Sean Welleck