Python Neural Networks - Tensorflow 2.0 Tutorial - Creating a Model

Key Takeaways

hey guys and welcome back to the third neural network tutorial now in today's video we're actually gonna be working with the neural network so we're gonna be setting up a model we're gonna be training that model we're gonna be testing that model to see how well it performed we will also use it to predict on individual images and all of that fun stuff so without further ado let's get started now the first thing that I want to do before we really get into actually writing any code is talk about the architecture of the neural network we're going to create now I always found in tutorials that I watch they never really explained exactly what the layers were doing what they looked like and why we chose such layers and that's what I'm hoping to give to you guys right now so if you remember before we know now that our images they come in essentially as like 28 by 28 pixels and the way that we have them is we have an array and we have another array inside it's like a two dimensional array and house pixel values so maybe it's like 0.1 0.3 which is the grayscale value and this goes and there's times 28 and each row of these these pixels now there's 28 rows obviously because while 28 by 28 pixels so in here again we have the same thing more pixel values and we go down 28 times right and that's what we have and that's what our array looks like now that's what our input data is that's fine but this isn't really gonna work well for our neural network what are we gonna do we're gonna have one neuron and we're just gonna pass this whole thing to it I don't think so that's not gonna work very well so what we need to actually do before we can even like start talking about the neural network is figure out a way that we can change this information into a way that we can give it to the neural network so what I'm actually gonna do and what I mean most people do is they they do what's called flatten the data so actually maybe we'll go I can't even go back once I clear it but flattening the data essentially is taking any like interior list so let's say we have a list like this and just like squishing them all together so rather than so let's say this is like 1 2 3 if we were to flatten this what we would do is while we remove all of these interior arrays or lists or whatever it is so we would just end up getting data it looks like 1 2 3 and this turns out to work just fine for us so in this instance we only had like one element in each array but when we're dealing with 28 elements in each sorry list listen array they're interchangeable just in case I keep saying those what will essentially have is what flat in the data so we get a list of length 784 and I believe that is because well I mean I know this house because 28 times 28 equals 784 so when we flatten that data so 28 rows of 28 pixels then we end up getting 784 pixels just one after each other and that's what we're gonna feed in as the input to our neural network so that means that our initial input layer is gonna look something like this we're gonna have a bunch of neurons and they're gonna go all the way down so we're gonna have 784 neurons so let's say this is 7 8 4 I know you could probably hardly read that but you get the point and this is our input layer now before we even talk about any kind of hidden layers let's talk about our output layer so what is our output well our output is gonna be a number between 0 & 9 ideally that's what we want so what we're actually gonna do for our output layer is rather than just having one neuron that we use kind of in the last the two videos ago as an example is we're actually gonna have 10 neurons each one representing one of these different classes right so we have 0 to 9 so obviously 10 neurons or 10 classes so let's have 10 neurons so 1 2 3 4 5 6 7 8 9 10 now what's gonna happen with these neurons is each one of them is going to have a value and that value is gonna represent how much the network thinks that it is each neuron so for example say we're classifying the image that looks like a t-shirt or maybe like a pair of pants so those are pretty easy to draw so let's say this is the image we're given a little pair of pants what's gonna happen is let's say pants is like this one like this is the one it actually should be all of these will be lit up a certain amount so essentially maybe we'll say like we think it's 0.05 percent this we have like a degree of certainty that it's 10 percent this one and then it is like we think it's 75 percent pants so what we'll do when we are looking at this output layer is essentially will define whatever one is the greatest so whatever probability is the greatest and then say that's the one that the network predicts is the class of the given object right so when we're training the network what we'll do essentially is we'll say okay well we're giving the pants so we know that this one should be one right this should be a hundred percent it should be one that's what it should be and all these other ones should be zero right because it should be a zero percent chance to say anything else because we know that it is pants and then the network will look at all this and adjust all the weights and biases accordingly so that we get it so that it lights this one up directly as one at least that's our goal right so once we do that so now we've talked with the input layer and the output layer now it's time to talk about our hidden layers so we could technically train a network that would just be two layers right and we just have all these inputs that go to some kind of outputs but that wouldn't really do much for us because essentially that would just mean we're just gonna look at all the pixels and based on that configuration of pixels will point to you know these output layers and that means we're only gonna have which I know it sounds only 784 times 10 weights and biases so 784 times 10 which means that we're only gonna have 7840 weights right weights and biases things to address so what we're actually gonna do is we're gonna add a hidden layer inside of here now you can kind of arbitrarily arbitrarily pick how many neurons you're gonna have in your hidden layer it's a good idea to kind of go off based on percentages from your input layer but what we're gonna have is we're gonna have a hidden layer and in this case this hidden layer is gonna have a hundred and twenty eight neurons so we'll say this is 128 and this is known as our hidden layer so what will happen now is we're gonna have our inputs connecting to the hidden layer so fully connected and then the hidden layer will be connected to all of our output neurons which will allow for much more complexity of our network because we're gonna have a ton more biases and a ton more weights connecting to this middle layer which maybe we'll be able to figure out some patterns like maybe to look for like a straight line that looks like a pant sleeve it looks like an arm sleeve maybe they'll look for concentration of a certain area in the picture right and that's what we're hoping that our hidden layer will maybe be able to do for us maybe pick on pick up on some kind of patterns and then maybe with these combination of patterns we can pick out what specific image it actually is now we don't really know what the hidden network or hidden layer is gonna do we just kind of have some hopes for it and by picking 128 neurons we're saying okay we're going to allow this Hinn layer to kind of figure its own way out and figure out some way of analyzing this image and then that's essentially what we're gonna do so if you have any questions about that please do not hesitate to ask but the hidden layers are pretty arbitrary sorry I just dropped my pen which means that you know you can kind of experiment with them kind of tweak with them there's some that are known to be to do well but typically when you're picking a hidden layer you pick one and you typically go at like maybe 15-20 percent of the input size but again it really depends on the application that you're you're using so let's now actually just start working with our data and creating a model so if we want to create a model the first thing that we need to do is define the architecture or the layers for our model and that's what we've just done so I'm gonna type it out fairly quickly here and again you guys will see how this works so I'm gonna say model equals in this case Cara's dot sequential believe that's how you spell it and then what we're gonna do is inside here put a list and we're gonna start defining our different layers Syrena side care apps dot layers and our first layer is gonna be an input layer but it's gonna be a flattened input layer and the input underscore shape is gonna be equal to 28 by 28 so remember I talked about that initially what we need to do is well we need to flatten our data so that it is passable to all those different neurons right so essentially I got misspelled shaped correct shape correctly so essentially whenever you're passing in information that's in like a 2d or 3d array you need to flatten that information so that you're gonna be able to pass it to an individual neuron as opposed to like sending a whole list into one neuron right now the next layer that we're gonna have is going to be what's known as a dense layer now a dense layer essentially just means a fully connected which means that what we've showed so far which is only fully connected neural networks that's what we're gonna have so each node or each neuron is connected to every other neuron in the next network so I'm going to say layers dense and in this case we're gonna give it a hundred twenty-eight neurons that's what we've talked about and we're gonna set the activation function which we talked about before as well to be rectified linear unit now again this activation function is somewhat arbitrary in the fact that you can kick different ones but rectifier linear unit is a very fast activation function and it works well for a variety of applications and that is why we are picking that now the next layer is gonna be another dense layer which means essentially another fully connected layer sorry and we're gonna have ten neurons and this is gonna be our output layer and we're gonna have an activation of softmax now what softmax does is exactly what i explained when showing you that kind of architecture picture it will pick values for each neuron so that all of those values add up to one so essentially it is like the probability of the network thinking it's a certain value so it's like I believe that it's 80% this 2% this 5% this but all of the neurons there those values will add up to one and that's what the softmax phone softmax function does so that actually means that we can look at the last layer and we can see the probability or what the network thinks for each given class and say maybe those are two classes that are like 45% each we can maybe tweak the output of the network to say like I am not sure rather than predicting a specific value right all right so now what we're gonna do is we're gonna just set up some parameters for our model so I'm gonna say model dot compile and in this case we're gonna use an optimizer of atom now I'm not really gonna talk about the optimizer Adam is typically like pretty standard especially for something like this we're gonna use the loss function of sparse and in this case underscore katz AGG oracle believe i spoke that correctly and then cross-entropy now if you're interested in what these do and how they work in terms like the math kind of side of them just look them up there's their very famous and popular and there again are somewhat arbitrary terms are how you pick them now when I do metrics I'm gonna say metrics equals accuracy and again this is just gonna define what we're looking at when we're testing the model in this case we care about the accuracy or how low we can get this loss function to be so yeah you guys can look these up there's tons of different loss functions some of them have different applications and typically when you're making a neural network your mess around with different loss functions different optimizers and in some cases different metrics so now it is actually time to train our model so to train our model what we're gonna do is model dot fit and when we fit it all we're gonna do is give it our train images and our train labels now we're gonna set the amount of epochs so now it's time to talk about epochs now epochs are actually fairly straightforward you've probably heard of the word epoch before but essentially it means how many times the model is gonna see this information so what an epoch is gonna do is it's gonna kind of randomly pick images and labels obviously correspond to each other and it's gonna feed that through the neural network so how many epochs you decide is how many times you're gonna see the same image so the reason we do this is because the order in which images come in will influence how parameters and things are tweaked with the network maybe seeing like 10 images that are pants is gonna tweak it differently than if it sees like a few better pants and a few that are a shirt and some that are sandals so this is a very simple explanation of how the epochs work but essentially it just is giving the same images in a different order and then maybe if it got one image wrong it's gonna see it again and be able to tweak and it's just a way to increase hopefully the accuracy of our model that being said giving more epochs does not always necessarily increase the accuracy of your model it's something that you kind of have to play with and anyone that does any machine learning or neural networks will tell you that they can't really like they don't know the exact number epoch they have to play with it and tweak it and see what gives them the best accuracy so anyways now it is time to actually well we can run this but let's first get some kind of output here so I'm gonna actually evaluate this model directly after we run it so that we can see how it works on our test data so right now what this is doing actually just training the model on our training data which means we're tweaking all the weights and biases we're applying those activation functions and we're defining like a mean function for the model but if we actually want to see how this works we can't really just test it on the training images and labels for the same reason I talked about before so we have to test it on the test images and the test labels and essentially see how many it gets correct so the way we do this is we're gonna say test underscore loss test underscore AC which stands for accuracy equals mall dot evaluate is that how you spell it maybe and then we're gonna do test images test underscore labels and I believe that is the last parameter yes it is so now if we want to see the accuracy of our model we can simply print out test underscore ACC and we'll just say like tested ACC just so we know because there is gonna be some other metrics that are going to be printing our test when we run this all right so now that we've done that let's actually run our file and see how this works so this is it this whole part here is all we actually need to do to create a neural network and do a model now actually let me just quickly say that this Karis not sequential what this does is it means a like a sequence of layers so you're justifying them in order where you say the first layer obviously is gonna be your input layer we're flattening the data then we're adding to dense layers which are fully connected to the input layer as well and that's what our model looks like and this is typically how you go about creating a neural network all right so let's run this now and see what we get so this will take a second or two to run just because obviously there is what we have 60,000 images in this data set so you know it's got a run through them it's doing all the epochs and you can see that we're getting metrics here on our accuracy and our loss now our test accuracy was 87% so you can see that it's actually slightly lower than what do you call it like the accuracy here oh it's the exact same oh it actually Auto tested on some data sets but anyways so essentially that is how this works you can see that the first five epochs which are these ones here ran and they increase typically with each epoch now again we could try like 10 epochs 20 bucks and see what it does but there is a point where the more epochs you do the actual like the less reliable your model becomes and you can see that our accuracy was started at 88.9 essentially and that was on like that's what it said our model accuracy was when we were training the model but then once we actually tested it which of these two lines here it was lower than the be tested or like the trained accuracy which shows you that you obviously have to be testing on different images because when we tested it here it said well it was 89% but then here we only got 87% right so let's do a quick tweak here and just see what we get maybe if we add like 10 epochs I don't think this will take a crazy long amount of time so we'll run this and see maybe if it makes a massive difference or if it starts leveling out or it starts going lower or whatnot let me let this run here for a second and obviously you can see the tweaked accuracy as we continue to go I'm interested to see here if we're gonna increase by much or if it's just kind of gonna stay at the same level all right so we're hitting about 90% and let's see here 91 okay so we got up to 91% but you can see that it was kind of diminishing returns as soon as we ended up getting to about 70 parks even yeah even like eight epochs after this we only increased by a marginal amount and our accuracy on the testing data was slightly better but again for the amount of epochs five extra epochs it did not give us a five times better result right so it's something you got to play with and see so anyways that has been it for this video in the next video I'm gonna continue using this model a little bit to actually predict on individual images I know I said I was gonna do that in this video but it's gotten a bit longer so let's move that into the next video if you guys enjoyed please make sure you leave a like and subscribe and I will see you again there [Music]