A.I. Learns to Play World's Hardest Game (QWOP)

Key Takeaways

[Music] welcome to the video in today's video I'm gonna be showing you how I built an AI agent to learn play and end up winning the popular flash game QWOP my inspiration for this game was seeing google's deepmind and how it's AI agent learned to walk and run from point A to point B and I match this with my deep deep frustration with the game cloth that I've had over the years as I've tried to master it manually with out any luck whatsoever I am absolutely trash at the game so I decided to cheat my way by creating an AI agent to play the game for me we are going to create our AI agent using an evolutionary algorithm or in other terms a genetic algorithm and what this algorithm does is it pretty much it's based off of evolution itself and how humans evolved from monkeys and it uses the same concepts of evolution to help our runner advance and learn how to beat the game so let's go ahead and kick off our genetic algorithm and I'll go through as it runs and explain what is happening and the algorithm itself first off we are using selenium to automate our agent and to automate the key presses of the runner the q wo and P so for the purpose of this video we are going to refer to our runner as Alan don't ask me why I chose the name Alan just a random name that popped into my head the algorithm that Alan is using to learn as he goes through the game is as follows we start by creating an initial population which is made up of genes and chromosomes linked together our genes for Alan are all the different combinations of key presses that are possible within the game plot using Q wo and P and then one gene called SL which is just kind of a sleep or pause don't press any buttons we then add a random timestamp between one tenth of second and three seconds to go along with that key press and that makes up the gene and then we string ten random genes together in a row to form the chromosome Alan will then continuously loop over this chromosome going through the key presses and times for each key press until he either falls over or wins and that is one individual within a population and a population is ten of these random chromosome combinations in a row and we just keep looping through the population and as we loop through we compute the fitness for each individual within the population and our fitness for Alan is the distance he has walked or ran and made it in the game and we get this score off of the screen by constantly taking screenshots of the game screen itself and extracting the text at the top where it says how far Alan has traveled and using that to just as the score for the fitness so each run for Alan until he either follows over or wins the game is one individual within the population so that randomly generated DNA that makes up a chromosome on repeat until he falls over or wins and then once that happens to a size population size of ten so ten runs of Alan dying or winning is a population size of ten and then we do this pseudocode in the repeat section so selection occurs we take the top two fitness scores the top two runs that Alan did so the distance that Alan made it the farthest we take those DNA of key presses and time press for each key and then we perform crossover crossover chooses a point within the chromosome in which it will cross over and swap the DNA from both of the two top scores and then it now makes child or offspring of the population which has a mix of those top two scores and those will now be the new population ten new offspring that will then be runs for Alan to try and complete and see if he does better and if he does better they become those new top scores that will then populate and reproduce to keep creating the next generations and generations so each time we repeat this step it is a generation because the DNA swaps we also perform mutation within the algorithm which where is theirs I give it a ten percent chance of randomly altering the DNA when it does this reproduction and offspring step if we fast forward to generation ten we can see that Alan has become extremely stubborn stubborn stubborn and he has learned to actually just take a knee almost and barely barely a minute over this is part of to do with nice for itself because it's purely just how far Allen has ran or at this pot crawled almost it doesn't take into account the speed or how fast he is moving so allen has learned to just take a knee and practically inch forward so props to Allen for being a stubborn stubborn runner and now we're gonna have to adjust our fitness X our Fitness score Fitness function to now be the score divided by the time the time it takes so we're gonna have to go ahead and rerun our simulation using this new updated fitness board so I'll speed up the clip speed and let's watch Allen evolve over time we can see off the bat that Allen is extremely stupid and hasn't learned anything at all he's but he's slowly trying out different combinations of key presses that ended up him falling over but eventually he starts to move forward direction and not start to and not look like a total fool he starts to kneel and actually be able to start making forward progress and if we fast-forward to generation 8 we can see that Allen has learned enough to scoot rapidly forward on one knee and make it all the way to 39 meters before face-planting into the ground and on the very next run Allen makes a break for it and actually encounter something that I didn't even know was in the game because I've never made it close this far the mythical hurdle and fails at hurdle damn it Alan Alan has to get back to the drawing board so that he can surpass this hurdle the next time he faces it Alan continues to run through the individuals within populations and continues to evolve over time so he just keeps doing trial and error learning from his mistakes so that he can have a better score and better run on the following the following run and you'll see that he slowly becomes faster and slowly becomes more intelligent as an AI agent in general this is the genetic algorithm at play and constantly repeating that loop of selection of the fittest natural selection mutation and reproduction to continuously get better over time and we can see that Alan is indeed learning this time when Allen faces the hurdle he's learned and does not fall down right away he's actually able to tip it over and then continuously move forward while pushing it but you'll see that still Allen has not learned to overcome the mythical hurdle and will actually fall over fast forward to generation 14 of the genetic algorithm and this time around Allen has evolved enough to actually surpass and beat the hurdle finally Congrats Allen and he is actually now going to be able to actually get himself over the hurdle after toppling it over which is a pretty impressive feat and he's learned this through the algorithm evolving and he will be able to get over the hurdle and hopefully make a push for the finish line [Applause] so as Allen makes the home stretch down the final 20 meters I just want to say thanks everybody for watching this video I hope you enjoyed it if you enjoyed it please drop a like and subscribe to the channel to see more a I coverage and more AI videos in the future it would mean a lot to me not to mention it would mean a lot to Allen I know he's become very attached to you all as you've been tchard with him through his evolution journey also let me know down in the comments whether you want to see a second video where I cover all the code used in this video to control Allen and the algorithm behind it Allen is coming up to the finish line and completing QWOP we successfully created an AI agent that evolved to beat the game [Applause] [Music]