OpenAI Blog

Measuring Goodhart’s law

Read the original on OpenAI Blog →

Goodhart’s law famously says: “When a measure becomes a target, it ceases to be a good measure. ” Although originally from economics, it’s something we have to grapple with at OpenAI when figuring out how to optimize objectives that are difficult or costly to measure.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at OpenAI Blog.

arXiv Machine Learning
Jul 31

Procedural Fairness in Multi-Agent Bandits

arXiv:2601. 10600v2 Announce Type: replace-cross Abstract: In the context of multi-agent multi-armed bandits (MA-MAB), fairness is often reduced to outcomes: maximizing welfare, reducing inequality, or balancing utilities.

By Joshua Caiata, Carter Blair, Kate Larson
arXiv AI
Aug 25

The Measurement Revolution? Credible Measurement and Inference in the Age of AI

The article discusses how artificial intelligence is reshaping measurement in economics by converting unstructured data into structured variables at low cost, enabling large‑scale measurement that was previously infeasible. It outlines three stages—discovery, construct definition, and observation—where AI impacts the measurement pipeline and stresses the importance of rigorous validation to ensure credible inference. The review offers guidance on navigating the shift from a single scalable measure to multiple plausible ones that can lead to differing empirical conclusions.

By Melissa Dell, Ashesh Rambachan
arXiv Machine Learning
Jun 9

Proper Calibeating

arXiv:2605. 26703v2 Announce Type: replace-cross Abstract: The classic concept of "calibrated forecasts" and its more recent refinement, "calibeating," are defined with respect to the standard quadratic scoring rule.

By Dean P. Foster, Sergiu Hart