HealthBench is a new evaluation benchmark for AI in healthcare which evaluates models in realistic scenarios. Built with input from 250+ physicians, it aims to provide a shared standard for model performance and safety in health.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at OpenAI Blog.
Introducing MentalHealthBench, a new benchmark developed by OpenAI, is designed to evaluate AI responses in realistic mental health conversations. The benchmark is expert-informed, focusing on assessing both helpfulness and safety of the AI’s replies. It aims to provide a standardized way to measure performance in sensitive mental health contexts.
arXiv:2603. 25821v2 Announce Type: replace-cross Abstract: We present Doctorina MedBench, a comprehensive evaluation framework for agent-based medical AI based on the simulation of realistic physician-patient interactions.
arXiv:2607. 15166v1 Announce Type: new Abstract: Most medical AI benchmarks measure whether a model knows the correct answer.
As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare applications. We introduce HealthAgentBench, a suite of 54 agentic healthcare tasks across 7 categories each with its unique environment.
Introducing GeneBench-Pro, a new benchmark testing AI performance in genomics, biology, and scientific research using complex, real-world datasets.
arXiv:2606. 31179v1 Announce Type: new Abstract: As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare applications.