Python Chat Bot Tutorial - AI ChatBot with Deep Learning (Part 4)

Key Takeaways

hey guys and welcome to the fourth video in the AI Champa series now in today's video we're actually gonna get into more of the fun stuff and get working with our model and getting it talking back and forth with us now this is not super difficult to do we've already trained the model there just a few things that we kind of need to set up here to get the model running now the first thing that we're gonna talk about is the fact that we don't really want to have to do all of this code every time that we want to use the model so every time we want to make one prediction in our current state we need to run all this code so we're gonna clean that up first we're gonna save the model in kind of a better form than this or just a different way and we're gonna make sure that we don't do all the pre-processing of this data multiple times now right now it only takes a few seconds to train the model but if you have a lot more data in this JSON file or more complex models then you definitely don't want to be re running this code if you don't have to so the first thing that I'm actually gonna do here is just set up a try except Clause and what we're gonna do is we're gonna try to open up some saved data which will save in just a second and if that doesn't work then we will generate the trick the data so we're gonna do try and then underneath the arena that put you accept and notice that I put if I spell accept correctly notice that I just put all of this code here so all the way up until training equals numpy dot array output equals numpy dot array in the except that's because we're not gonna do any of this if what's in the try works successfully now what we're gonna do is actually start by just importing Pikul up here and what we're gonna do is try to load in some pickled data and that data is gonna be our words our labels our training and our output data now that's because we need that data for the model so if we can have that model already or have that data already saved we've already processed it and used it before then there's no point in running all this into stemming the words and doing all that against what we're gonna do is just say with and and then in this case we're gonna say open and we're gonna put the name of where we're gonna save our information we haven't yet saved it I'm just gonna say data dot pickle and I believe you do have to end this in a pickle like dot pickle so do that and then we're gonna put eub or sorry not w BRB and that stands for read bytes because we're gonna save this data as bytes we're gonna do that as f and what we're gonna say is open data duck Pikul RB as f and we're gonna say what is it model or not model I gotta figure out how we're gonna do this we're gonna do words labels training output equals and then pickle load F now I know this probably doesn't really make much sense to you but what we're gonna do is we're gonna save all of these four variables into our pickle file and then when if it is saved and we open this correctly and it works fine we're gonna load in these lists because those are the only lists we actually need for our models so well pickle dot load that file it should load that up now obviously we haven't yet saved this so to save this what we're gonna do is at the end of this except here so right after training and output we're gonna save all of those in the same form so I'm actually just gonna copy this we're just gonna change a few things around right under here so let's forget the indentation correct we'll say with opened out pick 'l in this case we're gonna put it in WB mode and now what we're gonna do is instead of having we call it pickle dot load we're gonna say pickled dot dump we're gonna put all of these in brackets like this so it should be all of these in brackets like a tupple we're gonna do comma F and then close that bracket which means write all of these variables into a pickle file so we can save it now so this way when we run this code it's gonna try to open up this first if it can't open it it'll do all this and then it'll save it and then the next time that we run this it'll just simply open it up for us and we won't have to do that so now we have the model here we actually have to keep this model code like this but what we can do is just modify one thing so we're not training the model if we already have a model that exists so to do that what we're gonna do is say try and we're just gonna try to load the model and to load the models really easy which can do model dot load and then model dot TF learn just like that and now otherwise we'll say accept and we'll do all this so that's it for saving our model so now what's actually going to happen is when we run this script if a model already exists then we won't retrain the model and if this data that we've already pre processed exists and we've saved it as we already as data dot pickle we won't bother doing that as well now if you change anything in your intense JSON file just throw like like an X in here or something so that it doesn't open up your old pickle data it actually runs through all of this or you can just delete that we'll pick a file and delete the old model because you'll need to retrain it on that new information that you put in this intense JSON file so anyways that's it for that and now time to actually start making predictions now remember that when we trained our model we fed it bags of words so that is actually the same information we have to give our model if we want to make a prediction so the first step in kind of classifying any sentences or getting any kind of output from the model is to turn a sentence input from the user into a bag of words so I'm gonna write a function and this is called bag of words like this it's gonna take a let's just call it s and it's gonna take a list of words now this list of words is important because it needs to know how we're gonna create this bag of words which is going to be dependent on this words list now fortunately for us we have this words list we've loaded it in either with the pickle file where we've created it here so what we can do is just start writing this to create a bag of words so I'm gonna say bag equals a blank list that's where we're gonna store all the words obviously and then let me just go and write the rest I just gotta look at my other screen here so I'm gonna say s underscore words equals in this case NLT k dot word underscore tokenize and then we're just gonna put s in there so now we're gonna get a list of tokenized words and now we're gonna stem these words the same way we did before we're gonna say s underscore words equals and in this case we'll do a list and we'll say stemmer dot stem word lower or word in s underscore words that's gonna stem all of our words and now what we're gonna do actually is just modify this bag and then write a little bit more code so I'm gonna say bag actually in this case is gonna be 0 for underscore in range blend of words now what this is gonna do is just create a blank bag of words list and then we'll actually change the elements in here to represent if a word exists or if it doesn't such as setting up a list that has a bunch of zeros for however many words we have and now it's time to write a little for loop that will simply generate this bag list properly so now we're gonna say for s and I shouldn't can't do s we'll say for s II does that make sense to do that mmm not really well whatever will do for s e and s underscore words and now we're gonna say if we're sorry not F for I comma W in enumerate in enumerates and then words well we'll do now is say if W equals equals SP which means that the current word that we're looking at in this words list is equal to the word in our sentence what we will do is you will say bag I dot append and then we will simply append a 1 representing that the word exists otherwise we will do bag actually we don't need an else statement because we already filled them all with zeros so that's all we need to do to actually generate our bag of words I don't really want to go through this code it's pretty straightforward I hope you guys can understand that and then we're just gonna return a numpy don't array that has our bag like that now what this is gonna do is just exactly what it says here essentially take this bag of words convert it into a numpy ray and return it to well wherever we need that so now while we have this bag of words functions working fine and the next thing we're gonna do is write the code that will ask the user for some kind of sentence and then spit out a response so to do this will actually have to start using the model and I'm just gonna create a function called chat and we'll simply call chat at the end of our file if we want to start chatting with them all as opposed to just training it so now I'm gonna say print and this will just be like the first print statement like start talking with the bot just giving the user some input saying like the bots ready to go whatever something like that and now we're gonna say true let's say iymp which stands for input is equal to input and i'm just gonna put you : meaning like this is what you are saying like you type to the bot and then what I'm gonna do is I'm gonna say if I MP dot lower equals equals quits we'll put this in brackets then break so this is just a way that you can get out of the program if you type quit then it will simply break this while loop so that you can end because otherwise you would just keep going continuously or you'd have to close the program which we don't want so maybe let's add a thing here like type quit to stop so make that okay now after that what we're gonna do is say if they didn't type quit now we're gonna turn this input whatever words they typed in into a bag of words feed it to the model and get what the models response should be so to do this what we're gonna do is say what is it model dot predict and then in here we're gonna pass a list so just create a blank list like this and inside the list we're gonna say bag of words IM p : words now the reason we do this is because model dot predicts actually makes predictions on multiple things at once so it expects you to give it a bunch of different entries and then it gives you a bunch of different predictions but in our case we only want to predict for one thing but we still have to feed it in a list so we just put it in a blank list we say bag of words input words and that's our function bag of words will create a bag of words with the input that we gave it so we're gonna do that we're gonna save that as like results okay and we'll say results equals model dot predict like that now the issue that is when we get our results we're not gonna get any like real good output to give to the user all it's gonna give us is a probability and I'll show you what it looks like here so that you guys can get an idea if we want decide to print it out so let's run this now because I think this is important to show let's go Python me and go see if I rinse any errors by the way for now um hmm let's see it's run this one more time oh well I haven't called the chat function so that would probably be why that wasn't running let's see this now Python made up hi and it says u so let's type something let's type hello and int object has no attribute append hmm oh sorry my bad guys let's fix this first of all before I show that so bag I instead of dot append one is just gonna be equal to one so change that line right there we're gonna run Python main dot PI again and now if I type something like Hello you can see that this is our output for the models prediction now this doesn't really mean anything to us this is actually just a bunch of different probabilities and this is how probable the model thinks each of our different little neurons that are there is a what do you call it like how probable it thinks each neuron is that class essentially so we type hello it tries to classify hello and what it does says well I think it's this much this much this neuron I think it's this much this neuron it's this much this neuron telling you how likely it thinks it is each specific class because remember each neuron represents a specific class so what we need to do is actually pick out the greatest number in here and say that is the classification from our model so to do that there's a few little tricks we can use I'm just gonna use what's known as our max from numpy so to do that I'm going to say results underscore index equals numpy Arg max and we're gonna put results and what this will do is give us the index of the greatest value in our list so just like our list represents all of the different probabilities of classes or tags or intense or whatever we want to call them all of these things like greeting goodbye age whatever it'll give us the index of the greatest number then we can use that index to figure out which responds to actually display so now what we're gonna say is we'll say tag equals in this case labels and here what we'll just put is results underscore index now label stores all of the different labels so when we do this it'll give us the label that it thinks our message is so now let's try just printing the tag first of all and see what what we're getting from our model so let's finish running that let's go python maned up high and let's go hello and now we can see that it thinks it is a greeting if I say goodbye it thinks it is goodbye if I say like what are the hours it says hours right so it's giving us the tag that it thinks this intent is now so far it's worked fairly well let's actually start getting some responses though now we're just showing the tag but if we want to show a response we simply need to look through this JSON file and pick one of the responses that match with these tags so this will be the last part of the video I know this is a bit long but let's just power through and finish this real quick so to do this we're gonna open up the JSON file find that specific tag and then just pick a random response for it so JSON should be imported up here we're just gonna copy this open from the JSON file we're actually data is already there so we don't need to open it because it's already loaded in so we'll say for in this case tag in data intense now I mean I called the intense and the other one doesn't really matter actually let's change it to TG because we already have tag here so we'll say for TG and data intense and then we'll say if TD tag equals equals tag we'll say it responses equals and in this case it'll be TG and should be just responses and then what we can do is just say well now that we have responses like this for or we'll just randomly choose one of the responses and display them so to do that we'll say print random got choice responses now I believe I did this correctly I could have messed up a little bit we will see so for TG in data in tents that should loop through all of these dictionaries then we're gonna say for responses TG yeah that should work properly let's run this and let's try our chat bot and see if this is working now so Python maned up high let's go Tim or let's go hello and now you see good to see you good to see you again let's do goodbye we get hi there how can I help you obviously it didn't really understand what we were saying there maybe let's say what do you sell says we sell chocolate chip cookies for two dollars so you can see that clearly the chat pod is working and everything is going well so in the next video I'm just gonna show you a few little tweaks with the chat bot how we can change the intense file modify it I'm just some extra things but honestly this is how you use the chat bot if this is all you wanted here you go I hope you guys enjoyed the series if you're not gonna be watching the next video where I'm just gonna be showing you some fine-tuning and fixing and modifying and all of that stuff if you did please make sure you leave a like and subscribe to the channel and I will see you again in another video