Chat Bot With PyTorch - NLP And Deep Learning - Python Tutorial (Part 3)

Key Takeaways

hey guys and welcome back to the third part of our chat bot tutorial so last time we created our data set and now in this part we will continue and implement the actual PI torch models or deep learning neural nets and then the training pipeline so let's do this and before we start I noticed a little arrow here so here I have to use a comma instead of a dot of course and then in our chat data set class in the get item function here I call this index not short I DxO index so now we can continue and now we want to create the actual model so for this we create a new file so let's say train dot pi sorry not trained up i but model dot pi and here we import torch and import torch torch dot and n s and n and then we create a new class for our models so class neural net or whatever you want to call this and this must be derived from n n dot module and then we have to define the init which gets self and the input size and the hidden size and the number of classes so this will be a feed-forward neural net with two hidden layers and now if you have a look at the slides again here I have the feed-forward neural net which gets our bag of words as an input then we have one and neural one layer fully connected which has the number of different patterns as the input size and then the hidden layer and then another hidden layer and the output size must be the number of different classes and then we apply the softmax and get probabilities for each classes so let's do this so we want to create three linear layers by saying self l1 equals n n dot linear and this gets the input size and then the hiddenness sighs then let's copy this to x so our second layer has the hidden size as input and the hidden size also is output and then the third layer has the hidden size as input and the number of classes as output so input size and number of classes must be fixed but you can change the hidden size if you want to and then let's also create an activation function for in-between so we use a reloj activation function then we want to import the forward or implement the forward pass so this gets self and x and then we say out equal self dot l1 and this gets X first then we apply our activation function in between so we say out equals self dot re Lu which gets X now then we do the same thing two more times so we apply our second linear layer which gets out as an input and then puts out the next output then the next activation function and then the third linear layer so l3 and at the end we want to be careful so we don't want and activations no activation and no softmax because lai we apply the cross-entropy laws and this will already apply this for us so don't apply this here and simply return the output then so this is our model now we can go back to our training file and then create our model so here we say model equals torch sorry this will be our model so we have to import this so we say from model import neural net then we create this so we say neural net and then it gets the input size it gets the hidden size and it gets the output size output size so we have to define this here so we say our hidden size let's say our hidden size equals 8 as well so you can change this then the output size equals and this is the number of different classes or texts we have so the length of the text and the input size is the number or the length of each pack of words that we created and the bag of words has the same length as the all words array so we could either use the length of all words or we say the length of X train 0 so the first bag of words because they all have the same size this is the input size so now let's for example let's print the input size and let's print the length of all we hurts to see if this is correct then let's print the output size and let's print the length or let's print the whole tax array to see if this matches and then let's simply save this and run this so let's say python drain that pi and then we get an arrow here because of course i forgot to call this super method so here I have to call super neural-net and then it also gets self and then the super init function so let's clear this and run this again and now we see that this is working so we have the correct input size which matches the length of all words and we have the output size seven because we have seven texts in this array so this is working and let me delete this print statement because I don't want this anymore and now we can continue so one more thing that we can do is if we have GPU support so then we can use it so let's check if we have the cheap u available by saying device equals torch dot device and then we say this is the cuda device if torch dot cuda dot is available as a function otherwise we simply use the cpu and then we can push our model to the device if it is available and then let's create the loss and optimizer as always in our PI torch training pipeline so we say criterion equals n n dot cross entropy loss and our optimizer equals torch dot Optim dot atom and we want to optimize model dot para parameters and as a learning rate we say this is the learning rate that we have to specify so let's put them here to our hyper parameter so let's say learning rate equals point zero zero one and let's also define the number of epochs some epochs num epochs equals 1000 so you can try out different ones here and then let's do our actual training loop so we say for epoch in range num epochs and then we use our training loader so we say for and then we can unpack this words and labels in our training loader and then we also want to push it to the device so as we say words equals words dot to device and the same with the labels labels equals labels to device and then we call or do the forward pass by simply saying out out put equals and then we call model with words as the input and then let's calculate the loss or the loss equals the criterion which will get the predicted out and the actual labels then we do the backward pass the backward and optimizer steps so we have to empty the gradients first so this is one thing that we should remember in the PI torch optimizations we say optimizer dot zero zero grat then we call loss dot loss dot back sorry backwards to calculate the back propagation and then we say optimizer dot step and then this is our training loop and now it's also print some things so let's say if epoch plus 1 modulo 100 equals equals zero then so every 100th step then we want to print as an F string we want to print the current epoch which is the epoch plus 1 and we also want to print all epochs a number of epochs and we say loss equals and then here loss dot item and format it and to print let's say four point four F four decimal values then we are done and then let's also print let's grab this and print the final loss here so we say final loss equals the loss at the end and then let's clear this and run this and hope that everything is working so torch is not defined so here we say torch dot cuda so again let's run this and now our training loop starts so we see our loss is 0.7 in the next step it's already decreased alot so only point zero seven and it's further decreasing so now it's done and we could see that our loss successfully decreased with each epoch and at the end it's very low so our example patterns are not very complex but we can see that our neuron that is very good for this purpose so yeah I think that's it for part three and in the last part we will see how we can now save this start model or this train model and then use it in our chat function and then implement the actual chat so see you next time and if you liked this tutorial please like and subscribe