The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper introduces ReviewMap, a framework that distinguishes between two evaluative acts in peer‑review comments: assessment of claims and scholarly standards versus calibration of contribution value and venue fit. Using multi‑model LLM consensus on 500 ICLR 2026 papers, the act layer is reliably automatable (inter‑model κ ≥ 0.75) while the attribute layer still requires human oversight. The study also finds that 17% of reviewer‑paper instances lack panel‑coded calibration, with a higher calibration share in Strengths than Weaknesses, informing guidelines, rebuttal strategies, and discussion dashboards.
The paper introduces a verification‑centric framework and benchmark for evaluating large language models (LLMs) as assistants in peer review, focusing on error detection rather than imitation of human reviews. It presents a scalable benchmark that injects synthetic errors into conference papers to test LLMs’ ability to spot logical contradictions, and proposes a Multi‑Layered Review (MLR) framework that emphasizes deep manuscript comprehension before generating reviews. Experiments show the approach aligns well with human review scores, excels at error detection, and offers complementary reviewer insights, while also revealing vulnerabilities to adversarial manipulation.
arXiv:2605. 03202v2 Announce Type: replace Abstract: Large language models offer a tempting solution to address the peer review crisis.
arXiv:2608.21374v1 Announce Type: new Abstract: Literature reviews are essential to scientific progress, but rigorously evaluating automatically generated reviews remains difficult because many aspec...
arXiv:2506. 08134v4 Announce Type: replace Abstract: Peer review, the bedrock of scientific advancement in machine learning (ML), is strained by a crisis of scale.
arXiv:2609.23264v1 Announce Type: new Abstract: Peer-review evaluation is increasingly being automated with LLM-as-a-judge metrics, but this creates a measurement risk. A review may receive a high sc...