OpenAI Blog

AI and compute

Read the original on OpenAI Blog →

We’re releasing an analysis showing that since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3. 4-month doubling time (by comparison, Moore’s Law had a 2-year doubling period)[^footnote-correction].

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at OpenAI Blog.

OpenAI Blog
May 5, 2020

AI and efficiency

We’re releasing an analysis showing that since 2012 the amount of compute needed to train a neural net to the same performance on ImageNet classification has been decreasing by a factor of 2 every 16 months. Compared to 2012, it now takes 44 times less compute to train a neural network to the level of AlexNet (by contrast, Moore’s Law would yield an 11x cost improvement over this period).

OpenAI Blog
Aug 16, 2017

More on Dota 2

Our Dota 2 result shows that self-play can catapult the performance of machine learning systems from far below human level to superhuman, given sufficient compute. In the span of a month, our system went from barely matching a high-ranked player to beating the top pros and has continued to improve since then.

Microsoft Research
Sep 23

Offloaded inference for real-world physical AI robotics

Microsoft Research reports that offloading AI inference from robots to external hardware can enhance task success, increase efficiency, and enable more advanced physical AI workloads. The study suggests that moving inference beyond the robot’s onboard processors allows the hardware to keep pace with growing AI capabilities. This approach demonstrates a practical way to improve robotic performance in real-world settings.

By Ganesh Ananthanarayanan, Matthew Balkwill, Xenofon Foukas, Sanjeev Mehrotra, Bozidar Radunovic, Connor Settle, Ankit Verma, David White, Shawn Cicoria, Mark Martin, Rachel Johnson, Mayur Patel
arXiv Machine Learning
Sep 18

The Environmental Impacts of Language Model Training Keep Rising Now is the Time to Catch Impacts on the Rebound

The paper analyzes the environmental footprint of machine learning model training, focusing on large language models and their hardware. It finds that energy use and environmental impacts have risen exponentially over the past decade, even when employing carbon‑efficient electricity and more efficient hardware. The study argues that optimization strategies alone cannot curb these impacts due to a rebound effect, and stresses the need to evaluate hardware life‑cycle impacts and integrate environmental metrics into NLP research practices.

By Cl\'ement Morand (STL), Anne-Laure Ligozat (ENSIIE, LISN, STL), Aur\'elie N\'ev\'eol (STL, LISN)