OpenAI Five

Key Takeaways

Skoda is one of the most popular and challenging competitive video games ever playing dota means you have to coordinate and focus as a team of five last year we built a box to defeated the world's best players at the 1v1 mini game this year we want to beat the best proteins at the full game to do this we've built a new AI system which we spend the last few months training it uses a reinforcement learning at a much larger scale to train our BOTS to play together as a team we're now starting to play against amateur teams to test our skill and to our surprise so far we've won our first games against every team we've tested everyone to be everyone I can't do it but to figure out how good our bots really are yes and go to expert to take a look my name is William Lee better known in the DOTA community as blitz let's here with Krissy just gonna give a brief explanation of the game for people that don't know about the game rules are very simple it's kill the enemy team take the buildings every hero has a variety of unique spells they'll be playing a mirror mode where both teams have the exact same heroes here the human team is dealing significant damage to the base is 5 the crystal maiden comes in to defend the barracks she jumps in with blink dagger and shields herself with black King Bar while channeling freezing field to deal massive amounts of damage while being uninterruptible due to her combo she ends up taking up for human players in the 2v5 fight and Gigi's called game humans no longer think they can win they would be absolutely correct so their dev team got absolutely crushed I think is the fastest casting gig of my life then it went into game number two the humans kind of I mean they had time to like think about the game and stuff like that they got crushed even harder and the bots did exactly what I hope for is they own this area of the map you take away two thirds of the map they didn't even touch these two bottom towers and they would be 100% correct in this this is like one of the highest level plays that you can make this side of the map is incredibly hard for the boss to control and so they're just playing this top side and this mid side because they understand that these are the two most important parts to control the game the ability to like intuitively do this is insane doing it one game I can maybe chalk it up to just dumb luck doing it two games in a row flipping the sides means that it's more than just coincidence it took me and I'm fairly reasonably good at the game eight years before I learned some of the strategies I would say it was pretty easy to quantify for me it was about eight years for me to learn the strategies that the bot was intuitively doing to train our BOTS we use reinforcement learning myself play we run the game on over a hundred thousand CPUs and our bots learn from every game they play because dota is so complex to learn even for a single player we created a hyper parameter which we call team spirit the five bots start out completely selfish but tuning this knob tells them to care about their teammates so that they can learn to play together as one unit after seeing the bots win against the test team what's one to challenge it himself he has a higher rating from the test team and we paired him up with the best players in our audience and so we can see blitz about to die here blade says definitely dead yes so now the humans are in trouble truth the team members are are dead with 20 seconds remaining and the bots are about to take their first lane of rax another big fight happening here with the bus actually winning the game at first started to go to the humans but eventually I bought managed to beat this stronger team as well we're still far away from beating pro teams but I think everyone here was surprised to see this I think the teamfight aspect of the bot was excellent like it it didn't mess up when it came to coordination it was some of the best like just a pure team fighting because it felt like I was just getting like hammered every single time that I made a mistake and I feel like nobody needs don't do that dota World Championships will take place in August while the best players in the world aren't getting ready to compete we're also working on the next version of our bot to see how far we've come will host a live match in July well we'll play a team of top players overall but we're excited about is that the training method we use is very general we're focused on learning dota but we're hoping that this will give us more and more insight about how a I can solve complex problems anytime that's my favorite part about when I hear the Bob makes advancement I don't get scared I get excited because I think this is another thing for me to have to challenge myself to be this is something that not a lot of people will be able to do let's have at it you [Music]