OpenAI Five
📰 OpenAI News
OpenAI Five, a team of five neural networks, defeats amateur human teams at Dota 2 using self-play and reinforcement learning
Action Steps
- Train a neural network using self-play and reinforcement learning
- Use a scaled-up version of Proximal Policy Optimization running on multiple GPUs and CPU cores
- Implement a separate LSTM for each hero to learn recognizable strategies
- Benchmark progress by hosting matches against top players
Who Needs to Know This
AI engineers and researchers benefit from understanding the application of reinforcement learning and self-play in complex games like Dota 2, while product managers and entrepreneurs can learn from the potential of AI in esports
Key Insight
💡 Reinforcement learning can yield long-term planning with large but achievable scale without fundamental advances
Share This
🤖 OpenAI Five defeats amateur human teams at Dota 2 using self-play and reinforcement learning! 💻
Key Takeaways
OpenAI Five, a team of five neural networks, defeats amateur human teams at Dota 2 using self-play and reinforcement learning
Full Article
# OpenAI Five | OpenAI
[Skip to main content](https://openai.com/index/openai-five#main)
[](https://openai.com/)
* [Research](https://openai.com/research/index/)
* Products
* [Business](https://openai.com/business/)
* [Developers](https://openai.com/api/)
* [Company](https://openai.com/about/)
* [Foundation(opens in a new window)](https://openaifoundation.org/)
Log in[Try ChatGPT(opens in a new window)](https://chatgpt.com/?openaicom-did=04c0ad88-1922-4a36-b25d-deb133b57875&openaicom_referred=true)
* Research
* Products
* Business
* Developers
* Company
* [Foundation(opens in a new window)](https://openaifoundation.org/)
OpenAI Five | OpenAI
Table of contents
* [The problem](https://openai.com/index/openai-five#the-problem)
* [Our approach](https://openai.com/index/openai-five#our-approach)
* [Model structure](https://openai.com/index/openai-five#model-structure)
* [Exploration](https://openai.com/index/openai-five#exploration)
* [Coordination](https://openai.com/index/openai-five#coordination)
* [Rapid](https://openai.com/index/openai-five#rapid)
* [The games](https://openai.com/index/openai-five#the-games)
* [Differences versus humans](https://openai.com/index/openai-five#differences-versus-humans)
* [Surprising findings](https://openai.com/index/openai-five#surprising-findings)
* [What’s next](https://openai.com/index/openai-five#whats-next)
June 25, 2018
[Milestone](https://openai.com/research/index/milestone/)
# OpenAI Five
Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2.
Share
Our team of five neural networks, OpenAI Five, has started to[defeat](https://openai.com/index/openai-five/#thegames)amateur human teams at[Dota 2(opens in a new window)](http://www.dota2.com/play/). While today we play with[restrictions](https://openai.com/index/openai-five/#restricted), we aim to beat a team of top professionals at[The International(opens in a new window)](https://en.wikipedia.org/wiki/The_International_(Dota_2))in August subject only to a limited set of heroes. We may not succeed: Dota 2 is one of the most popular and[complex(opens in a new window)](https://purgegamers.true.io/g/dota-2-guide)esports games in the world, with creative and motivated professionals who[train(opens in a new window)](https://venturebeat.com/2017/02/12/dota-evil-geniuses/)year-round to earn part of Dota’s annual$40M[prize pool(opens in a new window)](https://www.esportsearnings.com/history/2017/games)(the largest of any esports game).
OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of[Proximal Policy Optimization](https://openai.com/index/openai-baselines-ppo/)running on 256 GPUs and 128,000 CPU cores—a larger-scale version of the system we built to play the much-simpler[solo variant](https://openai.com/index/dota-2/)of the game last year. Using a separate[LSTM(opens in a new window)](http://colah.github.io/posts/2015-08-Understanding-LSTMs/#lstm-networks)for each hero and no human data, it learns recognizable strategies. This indicates that[reinforcement learning(opens in a new window)](https://www.technologyreview.com/s/603501/10-breakthrough-technologies-2017-reinforcement-learning/)can yield long-term planning with large but achievable scale—without fundamental advances, contrary to our own expectations upon starting the project.
To benchmark our progress, we’ll host a match versus top players on August 5th.[Follow(opens in a new window)](https://www.twitch.tv/openai)us on Twitch to view the live broadcast, or[request(opens in a new window)](https://www.eventbrite.com/e/openai-five-benchmark-tickets-47144438284)an invi
[Skip to main content](https://openai.com/index/openai-five#main)
[](https://openai.com/)
* [Research](https://openai.com/research/index/)
* Products
* [Business](https://openai.com/business/)
* [Developers](https://openai.com/api/)
* [Company](https://openai.com/about/)
* [Foundation(opens in a new window)](https://openaifoundation.org/)
Log in[Try ChatGPT(opens in a new window)](https://chatgpt.com/?openaicom-did=04c0ad88-1922-4a36-b25d-deb133b57875&openaicom_referred=true)
* Research
* Products
* Business
* Developers
* Company
* [Foundation(opens in a new window)](https://openaifoundation.org/)
OpenAI Five | OpenAI
Table of contents
* [The problem](https://openai.com/index/openai-five#the-problem)
* [Our approach](https://openai.com/index/openai-five#our-approach)
* [Model structure](https://openai.com/index/openai-five#model-structure)
* [Exploration](https://openai.com/index/openai-five#exploration)
* [Coordination](https://openai.com/index/openai-five#coordination)
* [Rapid](https://openai.com/index/openai-five#rapid)
* [The games](https://openai.com/index/openai-five#the-games)
* [Differences versus humans](https://openai.com/index/openai-five#differences-versus-humans)
* [Surprising findings](https://openai.com/index/openai-five#surprising-findings)
* [What’s next](https://openai.com/index/openai-five#whats-next)
June 25, 2018
[Milestone](https://openai.com/research/index/milestone/)
# OpenAI Five
Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2.
Share
Our team of five neural networks, OpenAI Five, has started to[defeat](https://openai.com/index/openai-five/#thegames)amateur human teams at[Dota 2(opens in a new window)](http://www.dota2.com/play/). While today we play with[restrictions](https://openai.com/index/openai-five/#restricted), we aim to beat a team of top professionals at[The International(opens in a new window)](https://en.wikipedia.org/wiki/The_International_(Dota_2))in August subject only to a limited set of heroes. We may not succeed: Dota 2 is one of the most popular and[complex(opens in a new window)](https://purgegamers.true.io/g/dota-2-guide)esports games in the world, with creative and motivated professionals who[train(opens in a new window)](https://venturebeat.com/2017/02/12/dota-evil-geniuses/)year-round to earn part of Dota’s annual$40M[prize pool(opens in a new window)](https://www.esportsearnings.com/history/2017/games)(the largest of any esports game).
OpenAI Five plays 180 years worth of games against itself every day, learning via self-play. It trains using a scaled-up version of[Proximal Policy Optimization](https://openai.com/index/openai-baselines-ppo/)running on 256 GPUs and 128,000 CPU cores—a larger-scale version of the system we built to play the much-simpler[solo variant](https://openai.com/index/dota-2/)of the game last year. Using a separate[LSTM(opens in a new window)](http://colah.github.io/posts/2015-08-Understanding-LSTMs/#lstm-networks)for each hero and no human data, it learns recognizable strategies. This indicates that[reinforcement learning(opens in a new window)](https://www.technologyreview.com/s/603501/10-breakthrough-technologies-2017-reinforcement-learning/)can yield long-term planning with large but achievable scale—without fundamental advances, contrary to our own expectations upon starting the project.
To benchmark our progress, we’ll host a match versus top players on August 5th.[Follow(opens in a new window)](https://www.twitch.tv/openai)us on Twitch to view the live broadcast, or[request(opens in a new window)](https://www.eventbrite.com/e/openai-five-benchmark-tickets-47144438284)an invi
DeepCamp AI