arXiv Machine Learning
FrontierCS 2: A Benchmark for Computer Science R&D
Read the original on arXiv Machine Learning →FrontierCS 2 is a new benchmark comprising 48 open‑ended computer‑science research tasks that place language‑model agents inside a research environment, allowing them to submit incremental patches to a working repository. Each patch is scored by a judge on a development suite, while the final evaluation uses a held‑out suite and a private reference derived from the original authors’ code. The benchmark covers 13 domains, includes a private set of over 100 future tasks, and evaluates 12 agents, with Claude Opus 5.5 achieving the highest pass rate of 58%.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.