MIT News AI
Measure by measure, studying society accurately
Read the original on MIT News AI →The Flow has not summarised this story yet — read it at MIT News AI.
Related stories
OpenAI Blog
Apr 13, 2022
Measuring Goodhart’s law
Goodhart’s law famously says: “When a measure becomes a target, it ceases to be a good measure. ” Although originally from economics, it’s something we have to grapple with at OpenAI when figuring out how to optimize objectives that are difficult or costly to measure.
OpenAI Blog
Oct 9, 2025
Defining and evaluating political bias in LLMs
Learn how OpenAI evaluates political bias in ChatGPT through new real-world testing methods that improve objectivity and reduce bias.
MIT News AI
Jul 17
Following the questions where they lead
Assistant Professor Bailey Flanigan has arrived at complex computational methods for helping democracy thrive.
arXiv AI
Aug 18
Dynamic Evidence Collection Ecosystem for Assessment Integrity and Authentic Competence
arXiv:2608. 16016v1 Announce Type: cross Abstract: Generative Artificial Intelligence (GenAI) can produce high-quality essays, code, and design artefacts, challenging the validity of conventional assessments that rely on single-point submissions and product-only grading.
arXiv AI
Aug 12
The Epistemic Politics of AI Anthropomorphism
arXiv:2608. 00961v2 Announce Type: replace-cross Abstract: AI anthropomorphism is typically treated as a problem of user misperception requiring institutional correction.
arXiv Machine Learning
Sep 17
Making Political Text Scaling Comparable: Infrastructure and Hyperparameter Sensitivity for 17 Algorithms
The paper argues that computational text‑based ideal point estimation (CT‑IPE) methods should be viewed as configurable measurement pipelines rather than fixed estimators. It presents a large‑scale comparative experiment involving 17 CT‑IPE algorithms, 5,537 runs, and about 4.25 million left‑right position estimates, and describes shared infrastructure that enables joint execution of these heterogeneous methods. Sensitivity analyses reveal that most algorithms exhibit low hyperparameter sensitivity (ICC < .10), with any remaining sensitivity concentrated in a few key researcher choices such as the language or embedding model, seed keyword lists, and number of topics.