AI and compute
📰 OpenAI News
Compute used in largest AI training runs has increased exponentially since 2012 with a 3.4-month doubling time
Action Steps
- Track the growth of compute usage in AI training runs
- Analyze the impact of exponential growth on AI model development
- Prepare for continued advancements in compute capabilities to drive AI progress
Who Needs to Know This
AI engineers and researchers benefit from understanding the trend of compute usage in AI training, as it informs the development of more efficient models and allocation of computational resources
Key Insight
💡 Exponential growth in compute usage is a key driver of AI progress
Share This
💡 Compute usage in AI training runs doubles every 3.4 months!
Key Takeaways
Compute used in largest AI training runs has increased exponentially since 2012 with a 3.4-month doubling time
Full Article
We’re releasing an analysis showing that since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.4-month doubling time (by comparison, Moore’s Law had a 2-year doubling period)[^footnote-correction]. Since 2012, this metric has grown by more than 300,000x (a 2-year doubling period would yield only a 7x increase). Improvements in compute have been a key component of AI progress, so as long as this trend continues, it’s worth preparing for
DeepCamp AI