How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practices

The paper surveys how automated research systems—used for literature synthesis, ideation, workflows, writing, peer review, and end‑to‑end research—are evaluated. It compares task construction, evidence sources, evaluators, and scoring methods across six target areas, revealing that output checks, process checks, and human studies together provide a fuller picture. The authors identify gaps in diagnostic designs and offer reporting and audit recommendations to guide future evaluation efforts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• benchmarks