AI Teaches Itself to Play Flappy Bird - Using NEAT Python!

Key Takeaways

hey guys and welcome back so in today's video I want to show you a project I've been working on for the past couple days which is an AI teaching itself how to play flappy bird now I know I'm definitely not the first person to make this and I've taken a lot of inspiration from other youtubers like code bullet and seeing their videos and how they've tried to do this I took this as a challenge for myself trying to do this without looking at any of their code and you guys are gonna see how it turned out today so right now you're witnessing the AI train itself on how to play the game of flappy bird now the way that this works is that it starts off completely random having absolutely no idea what to do and how the game email operates and after many generations of slowly learning and slowly getting better it finally picks up on patterns and figures out what it can do to progress further in the level now this uses something called a genetic algorithm and the exact genetic algorithm I use is called neat now that stands for neuro evolution of augmenting topologies and we're gonna get into all those details later in the video and talk about exactly how it works but essentially after a few generations the AI starts to get exponentially better until eventually it reaches a point where it can't be beat and it can beat the level infinitely and keep going without ever hitting a pipe now to me this is super fascinating so I'm just gonna play a few clips and let you guys witness how this actually works [Music] so after about 13 generations you can see we've got a pretty good AI and chances are that this guy will be able to make it through the level infinitely without ever hitting a pipe now how does this work and how are we able to train an AI so quickly well that's what I'm gonna talk about now I'm gonna start explaining how this whole algorithm and everything works oh and after about an hour of running this program this is what the AI was able to accomplish a score of upwards of a thousand and I just ended up terminating the program at that point now before we can start to understand how the neat algorithm works we need to talk about neural networks now neural networks come in what we call layers and the first layer to talk about here is the input layer now the input layer is what is the information to our network it's what the network actually knows and what it sees you can almost think of it as the eyes of the AI now let's look at the frame on the right side of our screen here based on this frame we need some way in some pieces of information that we can feed to our neural network so that it knows what's going on now for me the three pieces of information I thought would be valuable is the position of the bird and the position of the two pipes or actually the distance between the bird the top pipe and the distance between the bird and the bottom pipe this way our AI has some idea where it is in the level and where the next pipes are that are coming up now once we have this input information we need some way to transport it into an output and this is where the output layer comes in now each neural network has both one input layer and one output layer and this output layer is gonna tell our AI what to do in our case we need to know whether we're gonna jump or not so we'll use one output neuron to represent the decision of jumping or not jumping and this is how we're gonna get our output from the neural network what we're gonna do is we're gonna give some values to our input so to our input neurons whatever those values are that we've decided then they're gonna pass that value somehow which I'm gonna talk about in a second to this output neuron and depending on the value of this output neuron we're either gonna jump or not jump alright so how do we pass these values well this is where we use something called connections and weights so essentially you know this very basic example each of our input neurons are gonna be connected to our output neuron using one connection so these lines that just showed up on the screen and each of these connections have what's called a weight now a weight is simply a number that represents how strong or how weak this connection is and these are the numbers that we're going to tweak to make our AI better or I guess make it worse in some cases so what we're going to do when we pass our values to our input neurons so our bird Y the top of the pipe and the bottom of the pipe is we're going to feed those through our neural network this is why it's called a feed-forward neural network they're going to have a weight applied to them and then they're gonna be passed to our output neuron where some more things can happen so we're gonna take what's called a weighted sum now weighted sum means we're gonna take each weight and multiply it by its corresponding input value so on input neuron 1 which is gonna be our bird Y position we're gonna take whatever that weight value is and multiply it by bird Y then we're gonna add it to whatever the distance between our bird and our pipe is multiplied by the next weight and so on and you can see that showing up right now now once that happens and we get those values at our output neuron we're going to apply two more operations to it and the first operation is something called a bias we're actually gonna add what's called a bias now a bias is simply a number that is going to allow us to kind of tweak our neural network in another way you can almost think of it as a y-intercept on a very basic graph this will just allow us to kind of move our network up and down a little bit and just shift it into the right position so these weights don't quite do the trick this bias should hopefully help us kind of adjust the network and and make it in the right dimensional space and I want to go too complicated with the math here so knees we're going to add this bias to our weighted sum and now we're going to apply what we call an activation function to this whole thing now this activation function simply allows us to get our value for this output neuron between two set numbers and this is really useful because it allows us to check whether or not you know and it's this number or its closest to this number so we know whether to jump or not to jump and in this case I'm using an activation function called tan H now there's tons of other ones you have sigmoid you have rel u you have all other kinds of activation functions but the tan h function is gonna squish whatever this value is in between the value of negative one and one so this is what the tan age function looks like you can see that the larger negative the number the closest negative the closer to negative one it's gonna be and the larger positive the number the closer to one it's gonna be and anything in between is kind of gonna be squished in between negative one and one and maybe kind of close to zero and that's the way that tan age works so based on this I'm actually able to now feed some values to my neural network get that output neuron see what it is and then compare that to whatever criteria I want so in this case I'm gonna say if this neural network has an output value of greater than 0.5 I'm gonna jump otherwise I'm not gonna jump and that's exactly what I've done in the code here to get this bird to actually move and to create this AF now you're probably wondering how we come up with these weights and biases and the answer is we don't we let the computer do it for us now what I've decided to use to do this is an algorithm called neat now there's lots of other ways to do this for a neural network but for a game like flappy bird this makes a lot of sense and you're gonna understand why after I explain it neat stands for a neuro evolution of augmenting topologies and essentially it's inspired on natural selection and natural selection is kind of a process of generations continually learning and getting better and better and better until eventually they get as good as they can get now that's exactly what we do here with our flappy birds as you guys have already seen through some of the simulations we're gonna start with an observation and that observation is we have no idea what the correct number for the weights and biases of our bird should be there's no way to kind of figure that out without doing tons of different tests so what we do is you say okay we don't know so we'll start by creating a population of birds and this population is gonna start off completely random and each population is going to consist of will a bird and a neural network that controls that bird now this neural network is gonna start with random weights and random biases and what we're gonna do is we're going to test all of these neural networks on our level or on our game and see how well they do we're going to evaluate what we call their fitness now fitness is different depending on what game you're playing in flappy bird the way that we determine how well a bird did is how far it progressed in the level you can do that in a few different ways but I've decided to just say every frame it moves forward without dying or without losing it gets another point and that's gonna be its fitness score so what we're gonna do is at the end of the simulation when all of our birds have died we're gonna see which one's performed the best in this set of population we're going to take those birds a certain percentage of them and we're gonna breed them and mutate them to create a brand new population of a hundred new birds and we're gonna get rid of all of the other birds that did poorly so now what we have is we have some offspring from the best birds from the last generation and we hope that these birds are gonna perform better than the previous generation because they come from the best of that previous generation now if you're interested in learning how neat actually breeds and mutates these neural networks that I have left a link down below to the original neat paper it's worth noting that I'm oversimplifying almost everything here so if you're interested in learning more about the technical and specific details I definitely recommend you check that out now this has been a little bit of a different video than what I usually do and if you guys like this style of presentation and of teaching definitely let me know in the comments down below because I can start implementing it more into the channel now with that being said this video is taking a crazy amount of time to make so I'd really appreciate it that if you enjoyed it and if you learned a little bit you hit the like button down below and even subscribe to the channel if you're not already with that being said I post all kinds of Python videos and this entire project was actually programmed using Python I used a module called PI game and a pipe Python module called neat Python which is actually how I did all of this AI and if you go ahead and look at the code down below using my github link there you'll see that this is actually pretty simple to do in that it's not crazy complicated there's a few things you have to tweak and mess with and it was a bit of a learning curve for me but if you're interesting getting into AI and kind of seeing how this works I will probably be making a tutorial about this in the future so make sure you're subscribed and stay tuned for that so finally if you guys enjoyed the video please please please hit the thumbs up button down below it really helps me out and even just leave a comment and let me know what you most like to bet or what you want to see in the future on the channel so with that being said go follow my Twitter go follow my Instagram join the discord and I will see you guys in another video