Benchmarks and evaluation

DABStep: Data Agent Benchmark for Multi-step Reasoning

CO₂ Emissions and Models Performance: Insights from the Open LLM Leaderboard

Benchmarking Language Model Performance on 5th Gen Xeon at GCP

Rethinking LLM Evaluation with 3C3H: AraGen Benchmark and Leaderboard

Introducing the Open Leaderboard for Japanese LLMs!

Judge Arena: Benchmarking LLMs as Evaluators

Introducing SimpleQA

A factuality benchmark called SimpleQA that measures the ability for language models to answer short, fact-seeking questions.

MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

We introduce MLE-bench, a benchmark for measuring how well AI agents perform at machine learning engineering.

Introducing the Open FinLLM Leaderboard

Our Transformers Code Agent beats the GAIA benchmark 🏅

BigCodeBench: The Next Generation of HumanEval

Launching the Artificial Analysis Text to Image Leaderboard & Arena

Benchmarking Text Generation Inference

Introducing the Open Arabic LLM Leaderboard

Introducing the Open Leaderboard for Hebrew LLMs!

Bringing the Artificial Analysis LLM Performance Leaderboard to Hugging Face

Introducing the Open Chain of Thought Leaderboard

The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare

Introducing the LiveCodeBench Leaderboard - Holistic and Contamination-Free Evaluation of Code LLMs

Generative AI to quantify uncertainty in weather forecasting

Posted by Lizao (Larry) Li, Software Engineer, and Rob Carver, Research Scientist, Google Research Accurate weather forecasts can have a direct impact on people’s lives, from helping make routine decisions, like what to pack for a day’s activities, to informing urgent actions, for example, protecting people in the face of hazardous weather conditions. The importance of accurate and timely weather forecasts will only increase as the climate changes.