AI-assisted peer review is increasingly discussed and adopted as a tool to support the scientific publishing process, yet there is little systematic understanding of how publication venues regulate its use or of how capable current AI review systems are. We address these questions by first surveying reviewer-facing AI policies across 111 leading AI/NLP conferences and medical journals, revealing substantial regulation differences between the two communities.
arXiv:2608. 03581v1 Announce Type: cross Abstract: AI-assisted peer review is increasingly discussed and adopted as a tool to support the scientific publishing process, yet there is little systematic understanding of how publication venues regulate its use or of how capable current AI review systems are.
arXiv:2506. 08134v4 Announce Type: replace Abstract: Peer review, the bedrock of scientific advancement in machine learning (ML), is strained by a crisis of scale.
arXiv:2605. 03202v2 Announce Type: replace Abstract: Large language models offer a tempting solution to address the peer review crisis.
arXiv:2609.05947v1 Announce Type: new Abstract: Peer review is central to quality control in science. However, existing evaluations of AI-assisted peer review mainly focus on the overall quality of g...
arXiv:2606. 10159v1 Announce Type: cross Abstract: AI is increasingly used to support scientific peer review, from manuscript screening, reviewer assistance to editorial triage.
arXiv:2603. 20450v2 Announce Type: replace-cross Abstract: A number of scientific conferences and journals have recently enacted policies that prohibit LLM usage by peer reviewers, except for polishing, paraphrasing, and grammar correction of otherwise human-written reviews.
arXiv:2609.23264v1 Announce Type: new Abstract: Peer-review evaluation is increasingly being automated with LLM-as-a-judge metrics, but this creates a measurement risk. A review may receive a high sc...
FIRSTPASS is a large-scale peer review dataset that captures complete multi-round editorial dialogues from a multidisciplinary high-impact journal, Nature Communications. It contains 3,668 records across five scientific domains—biology, chemistry, neuroscience, physics, and earth science—encompassing initial referee reports, author responses, and updated reviewer assessments. Each record is labeled with an outcome (STANDARD or EXTENDED) based on editorial decisions, and the dataset includes detailed parsing pipelines and evaluation scripts for reproducible AI benchmarking.
arXiv:2608. 14625v1 Announce Type: cross Abstract: Academic peer review is under mounting strain: NeurIPS 2025 received 21,575 submissions, ICLR 2025 received 11,603, and ICML 2025 received 12,107.
arXiv:2607. 22553v1 Announce Type: cross Abstract: Peer review is an essential process in scientific research, yet the growing workload has made its automation increasingly necessary.
arXiv:2608. 03659v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to generate scientific reviews, yet existing evaluations rarely examine whether different providers align with both conference decisions and human reviewing priorities within the same controlled setting.
Large language models are increasingly used as automated reviewers in scientific evaluation, creating a recursive feedback loop where later reviewers learn from earlier model-generated judgments. A study using Llama 3.1 8B fine‑tuned on ICLR reviews shows that incorporating synthetic reviews compresses rating distributions and reduces semantic diversity, a phenomenon termed scientific‑judgment collapse. To counter this, the authors introduce TrustReviewer, an open‑source LLM system that curates training data and applies paired activation steering at test time to preserve judgment diversity and improve recommendation alignment.