arXiv Machine Learning
Metag: A dataset to build agentic meta-reviewing capabilities
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
Oct 15
ProReviewer: Evidence-Driven Investigation for Scientific Peer Review
arXiv:2606.13349v2 Announce Type: replace Abstract: Effective peer review requires adaptive investigation that responds to evidence emerging from each paper. Existing LLM-based reviewers largely orga...
arXiv Computation and Language
2d ago
Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revision
The paper introduces Mr Dre, an evaluation suite that tests Deep Research Agents (DRAs) on multi-turn report revision, a process more reflective of human research practices. It defines a unified long‑form report protocol covering comprehensiveness, factuality, and presentation, and includes a human‑verified feedback simulation pipeline. Analysis of five DRAs shows that while agents can address most user feedback, they regress on 16‑28% of previously covered content and citation quality, and over successive turns they disrupt unrelated content and fail to preserve earlier edits, with prompt engineering and dedicated subagents offering limited improvement.
arXiv AI
Jun 30
meta-pipe: An LLM-agent pipeline for end-to-end automated systematic review and meta-analysis
arXiv:2606. 28363v1 Announce Type: cross Abstract: Objective: To describe the architecture and design rationale of meta-pipe, an open-source large language model (LLM)-agent pipeline that integrates the complete systematic review and meta-analysis (SR/MA) workflow -- from literature search through statistical analysis, manuscript generation, and quality assurance -- with mandatory human oversight at critical decision points.
arXiv Computation and Language
2d ago
Can AI Reviewers See Your Efforts to Improve Your Paper? A Large-Scale Study of Paper Revision Trajectories
The study examines whether AI reviewers’ scores reflect authors’ revisions by analyzing 6,959 paper trajectories from five AI conferences. Major revisions that add empirical evidence and improve framing are identified, yet most reviewers do not change scores for these improvements and agreement on score direction is weak. Reviewer-specific associations exist, but overall predictive power is limited, and brief negative comments can negate any scoring gains from substantive revisions.
arXiv Computation and Language
Sep 22
When Evidence Conflicts: Reliability-aware Meta-review Generation
arXiv:2609.24028v1 Announce Type: new Abstract: Generating coherent meta-reviews from multiple peer reviews is challenging when reviewer evidence conflicts and varies in reliability. Existing approac...
arXiv Computation and Language
Sep 16
PaperDoctor: Evidence-Grounded and Actionable Feedback for Scientific Papers in Progress
PaperDoctor is an agent framework that provides evidence‑grounded, actionable feedback for scientific papers before submission. It evaluates writing, layout, references, code, theory, prior work, and experiments through a three‑layer hierarchical system, linking each critique to specific evidence and revision suggestions. The system selectively rebuilds and reruns experiments to uncover reproducibility gaps, and an interactive interface lets authors explore findings tied to their manuscript.