Introducing LifeSciBench, an expert-authored, expert-reviewed benchmark for evaluating how AI systems handle real-world life science research tasks and decisions.
Sarah Friar, CFO of OpenAI, introduces a practical AI scorecard to measure ROI through useful work, cost per successful task, dependability, and return on compute.
OpenAI introduces BioBasicsBench, a 100-task benchmark measuring how well AI agents interpret raw experimental data and handle everyday wet-lab science.
Introducing MentalHealthBench, a new benchmark developed by OpenAI, is designed to evaluate AI responses in realistic mental health conversations. The benchmark is expert-informed, focusing on assessing both helpfulness and safety of the AI’s replies. It aims to provide a standardized way to measure performance in sensitive mental health contexts.
arXiv:2606. 26494v1 Announce Type: new Abstract: Medical AI remains organized around isolated models, whereas clinical care requires accountable capabilities that persist across time.
By Tianhan Xu, Lei Bao, Yongxiang Wang
arXiv:2606. 15708v1 Announce Type: new Abstract: Welcome to the ninth edition of the AI Index report.
By Sha Sajadieh, Loredana Fattorini, Raymond Perrault, Yolanda Gil, Vanessa Parli, Lapo Santarlasci, Juan Pava, Nestor Maslej, Russ Altman, Erik Brynjolfsson, Carla Brodley, Jack Clark, Virginia Dignum, Vipin Kumar, James Landay, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Elham Tabassi, Russell Wald, Toby Walsh, Dan Weld