Python Chat Bot Tutorial - Chatbot with Deep Learning (Part 3)

Key Takeaways

hey guys and welcome back to part three of AI chat bots in Python now up until this point all we've been doing is pre-processing our data and we have a few small things we need to do before we can move on from that but after that what we're gonna do is actually talk about the model that we're gonna use to make predictions based on a string of text and how that kind of works and just talk about that and draw some things I don't understand that as well as coded here now the first thing we need to do just a few minor errors here always run into typos and stuff when I'm doing this and I realize later but up here where it says Doc's X dot append pattern we need to change that to words and that is because we want to append the tokenized words I thought when we do this creating bag of words it actually works we also need to change this here to be labels because before that was classes and I don't have I don't have a list called classes so this needs to be labels I don't know why I called it classes and then the last thing is for these unique words here for W in words what we're actually gonna do is we're gonna say just add an if statement at the end of this just to remove any question marks because question marks are is a pretty common thing that people could type and we don't actually want that to have any meaning to the model so we're just gonna remove it so to do that we're gonna say if W not in and in this case question mark now we can do not in question mark where we just say W does not equal question mark doesn't really matter it's the same thing so we'll do does not equal question mark so those are the few fixes we need to do and now what we're gonna do actually is just change our output and our training into numpy arrays and that's because that is the form that needs to be taken by our model so we're just gonna say training equals an umpire dot array and in this case we're gonna say training and then we can just copy the same thing here and we'll do output equals numpy dot array and in this case we will say output now what this is gonna do is just take these arrays and change them or take these lists and change them into arrays so that we can feed them to our model so now that we've done that what we're gonna do is actually start building our model using T learn now this is very similar to tensorflow so if you've done tensorflow you understand how this works but I'm gonna code the model out and then we're going to kind of draw it and visualize what it actually looks like and understand how it's gonna work on classifying our data so we're gonna say TF which actually we have to type tensorflow because I didn't do import as - yep we'll do it tensor flowed reset underscore default underscore graph now we're just doing this to make sure that we get rid of all like previous settings and stuff it's resetting the underlying data graph or graph data you don't really have to understand what that means we're gonna say net equals in this case TF learned dot input underscore data and this K Swedish it's a shape equals and then none and the length of our training zero now what this is gonna do is define the input shape that we're expecting for our model so in this case we're getting the length of training zero because each training input is gonna be the same length so by doing this we're saying okay well the model should expect us to have an array of length forty five or however many words that we have right now next what we're gonna do is say next equals TF learn dot fully underscore connect it it's a lot of typing but it's we won't have to do that much I'm gonna say net eight and what this means is we're gonna say where you add this fully connected layer to our neural network which starts at this input data and we're gonna have eight neurons for that hidden layer or this first yeah I guess hidden layer now after this we're just gonna copy this again because we're gonna have another hidden layer that has eight neurons as well and then finally we need two more layers so we're gonna do TF learned up fully connected in this case we're gonna do net but it's gonna be our output layer so it's gonna be the length of output 0 and then what we're gonna say here is activation equals softmax now what this is gonna do essentially is allow us to get probabilities for each output and we'll talk about this morning when I draw the model but essentially softmax is gonna go through and give us a probability for each neuron in this lair and that will be our output for the network now after that we're just gonna add next equals TF learned regression regression and we're going to apply that to network now to train our model what we're gonna do is you say model equals in this case TF learned DN n prints a net and that's all we need to do now this is actually our complete model that's the whole a I kind of aspect of this I know it seems really short but let's go through exactly what I just typed and what this is now essentially we should start with an input data which is the length of our training data that we have two hidden layers with eight neurons fully connected I also connected to an output layer that has neurons representing each of our classes so let me actually bring out my drawing time but quickly and show you a little picture of this maybe make it a bit more clear on what's going on here so give me one second just to get this set up here and let's load up our little drawing thing if I can get this going here okay so what we have right now is we have a neural network and I'm just gonna draw it out for us we can have a look at exactly what it is like now we start with a a bunch of input neurons which are the length of our input data which means however many words we have because our bag of words is gonna be how many words we had in the thing so in this case I think we have something like 45 or something so we're gonna say we have a bunch of neurons and in this case let's just say we have like 45 of them okay so say this is like 45 now this is our input okay this is our first layer this is our input layer now the next layer that we have is 8 neuron so one two three four five six seven eight and they are connected to each our inputs so each input connects to each neuron just like this now I don't want to draw all of them out but you guys get the point it's fully connected just like that okay that's all I'm gonna draw for that now we have another layer that has eight neurons so we draw another eight neurons and all of these neurons are connected together fully connected once again so each one of these neurons connects to each other one in the lair again I'm not going to draw all the lines and then finally we have our output layer which has a soft max activation and this I'll just make it green so it's a bit different has six nerves so one two three four five six now again these are fully connected so it goes like this and all of them connect not gonna draw them all out so our output layer is special because it has this soft max activation so essentially what this means is all of these neurons are gonna be run through this softmax activation function and what that's gonna do is give a probability to each of these neurons so let's say that this first neuron represents hello okay and maybe that's the tag we have maybe this one represents goodbye and so on you know these all represent specific classes well if the model thinks that our response should be the hello tag so one of the responses under the hello tag the miss neuron will have a higher probability than all of these ones and that's essentially the way that this works it's gonna say well I think that you know it's 70% the hello tag so since that's the highest probability we will take that and well then we will grab some kind of response from Hello and spit that out to the user so all our model is really doing is predicting which tag that we should take a response from to give to the user so again we have six tags are because that's how many labels we have and we just our model picks one of these it actually gives us prediction values for all of them we say whatever one is the most highly predicted so maybe this one is like 90% this one's like 0.1% this one's like 5% we take the greatest highest predicted one we grab some responses from that we randomly pick one and then we give that to our user and that's kind of the way that this model works so we take in as input a bag of words and as output we get some kind of class or sorry label telling us what we think we should respond with what tag it comes from and that is how this very basic model works so what we are hoping is gonna happen when we start training and feeding information into our model so these hidden layers are gonna kind of figure out you know what words represent what one of these outputs may be if it sees the word hi it's gonna start changing some weights and changing some biases in here so that we get hello more commonly from that and these hidden layers are really what is doing kind of all of the work and they're what's gonna figure out what's going on and how this works and all of that kind of stuff with more and more intense or tags you would probably want to add more neurons to your hidden layers but two hidden layers is typically enough for a problem like this and you guys will see when we train the model this is actually a very accurate model for this kind of classification task that we're doing because essentially all we're doing is classifying sentences of words to some kind of output in some kind of tag so that is essentially how that works let's close that up now I hope you guys have a little bit of an understanding again I'm not gonna really teach neural networks but I want you guys to understand a little bit on why I chose this model and how it kind of works now the DNN is just a type of neural network and it's just gonna take these networks in or this network that we've defined here and just use that so now it's time to actually fit or our model so to do this we're gonna say model dot fit and what this means is we're actually gonna start passing it all of our training data so let's do this so we're gonna pass it training we're gonna pass it output we're gonna say number of underscore epochs and number of epochs is the amount of times that it's gonna see the same data so in this case we're gonna show it the same data a thousand times and hopefully the more it sees the data the better it gets at classifying now mess with this number make it 2,000 make it 5,000 make it a hundred and see what you get by doing that a lot of machine learning is trial and error so you got to understand that okay so batch underscore size we're gonna set as eight and we're gonna say show underscore metric equals true and this is just so that we get a nice kind of output when we're fitting this model now the last thing to do after we do this is simply save the model so we're just gonna say model dot Save and in this case we're just gonna save it as I guess we'll do like model dot tf2 learn and that's fine we'll save the model and that should just work for us when we start running this if we want to use the model to make some predictions we're gonna do that in the next video but let's just see how this works so let me go ahead I've already activated my environment I'm just gonna run this script and make sure that nothing went wrong here okay so there we go we're running our model you can see it's going through all these different epochs and you'll notice that when it stomps we had an accuracy of 99.97% which means it worked very well for our intents now again this is very simple because we only have six kind of tags here in our intents if you were to add more you would expect your accuracy would drop slightly but this model seems to be working very well at least on the data that it seems so far so the next step and in the next video we're going to start predicting data using this model and then in the final video in this series what we're going to do is actually set up a framework that will allow users to type to the model and get responses there's another kind of bonus part that I might do in this a sixth episode of this series but that's gonna have to wait till later so anyways that has been it for this chatbot AI video if you guys enjoyed please make sure leave a like and subscribe to the channel and I will see you in the next tutorial