Pytorch TutoriaL 3-How To Perform BackPropogation Using Pytorch

Key Takeaways

hello all my name is Krishna ham welcome to my youtube channel so guys today in this video we are actually going to see how we can perform back propagation using PI torch now here I'm just going to take a simple example guys and if you know if you have a basic knowledge about deep learning the most important thing in deep learning whenever you are creating an A + n or CNN or RN n there is always lot of front propagation that is forward propagation and backward propagation in the backward propagation we usually find out the derivatives or slopes so let me just tell you what exactly we are actually going to discuss today so in this back propagation we need to compute the derivatives right so we need to compute the derivatives and PI Tosh has a wonderful way actually to compute derivatives they have some inbuilt functions so suppose if I give you some examples of derivatives suppose guys my equation is like X to the power of two okay suppose this is my equation and if I try to find out the derivative of Y that is dy by DX that is derivative of Y with respect to X this particular value will be actually 2 - 2 multiplied by X this basically says that I can also write if we have something like this and probably everybody if you are a mathematics students guys you should be knowing that if X to the power of his n is there okay if it if it is given by this particular equation if Y is basically given by this particular equation and if we perform the derivative of this right then it usually becomes n multiplied by X to the power of n minus 1 right which is something like this so similarly over here you can see when I have Y is equal to X to the power of 2 then derivative of Y with respect to X will nothing be that will be 2 multiplied by X to the power of 2 minus 1 that is nothing but 1 right so this derivative is pretty much important because in deep learning if you go and see my complete deep learning playlist guys the link is given in the description there I actually explain you about forward propagation backward propagation chain rule and many more things and we know that from the input to the hidden layers first of all we multiply all the inputs with the way it's plus bias and then in the backpropagation we try to find out the derivative of this right considering the chain rule so in this video we are going to discuss both the chain rule part also end of simple derivative by just using this kind of equations so that we will try to understand how pi toss actually solves it so let's go ahead so over here first of all I am going to import torch which is the PI touch library itself and probably in my tutorial two and tutorial one I've already shown you that how you can install the torch library now suppose this is basically my equation y is equal to X square okay initially and if you know in a n n we usually have a question like Y is equal to WX plus B but here I've just taken in this particular equation how to actually do the back propagation in this I'll be showing you with the help of torch so let's go ahead so first of all I'm just going to define X so x over here will be nothing but a tensor okay and remember guys this is pretty much important needs to be defined as a tensor and there is also one more parameter suppose if I define this value as four point two and if I press shift tab over here there is one parameter which is called as required underscore grad okay this basically says that the gradient should be initialized when we are keeping this particular parameter as true then only we will be able to do the back propagation and we will be able to basically keep a reminder on that X variable let how the values are actually changing so here I'm actually going to take this as truth okay so if I go and see my X over here so here you can see that it is a tensor of four now I'm going to just use this particular equation that is y is equal to X to the power of two right so if I execute this and if I try to print Y it is nothing but it is showing me as a tensor of 16 now this is pretty much important guys what I have done over here I have initialized X with 4 so 4 to the power of 2 will be nothing but it will be actually 16 now let us go ahead and try to see how we can actually perform the back propagation and remember guys if I try to perform the back propagation if I try to perform the back propagation that equation will actually be back propagation if I try to perform or propagation if I try to perform the equation will become Y is equal to 2 multiplied by X right so this is what it will become so if I want to perform the back propagation with the help of torch we just need to write something like this so I'll use this Y and I'll be using dot and if I go and see if I just press tab you will be seeing that there is something called as backward okay so this backward if I try to see the definition guys if I press shift up it says that it computes the gradient of the current tensor gradient basically means the back propagation part the derivative part it actually tries to compute now if I try to execute this so you you'll be seeing that it has got executed perfectly now if I want to see see after the back propagation what will happen the equation will come Y is equal to 2 2 multiplied by X I know what is X over here X is nothing but 4 so Y will be equal to 8 so if I were really want to find out what is the output I can basically write sorry I can basically write print my X dot grad now if I go and see this you will be able to see that my value is actually coming as 8 this basically indicates that after I did the back propagation the derivative that I'm actually getting after I applied the derivative into this particular equation I'm getting the value as 8 this is a pretty simple way how the back propagation is actually done by using this particular example now why this is important guys because when we will be developing the a n n that is artificial neural network from scratch we will be using this particular function and this particular variable a lot ok we will basically be using a lot now let's go ahead and try to create more complex tensors you know because in our neural network when we are considering our a n n right we will be having many hidden nodes will be having various inputs right and within that there will be a lot of matrixes so all the time our X will just not be a single value instead it may be a matrix of values so what I'll do is that I will just create one weight weight variable again X is equal to or let me just create that this is my torch underscore input ok and Here I am actually go to take torch dot tensor and instead of just taking a single value now let me just define a list first of all I'll define a list which is actually a multi nested list suppose this is basically my list I will define my list and in this particular list I will write some elements like 2 3 comma 1 sorry 2 comma 3 comma 1 okay comma suppose I will try to write 4 comma 5 comma 3 okay and this is basically a multi nested list and if I try to convert this into array then it will become a two-dimensional array itself right suppose if I say 7 comma 6 comma 4 right so these are all my elements over here and what I am doing is that I am actually using this list over here and again I will try to use the same parameter which I used over here as required Grad is equal to 2 because this will be pretty much required guys over here too right once I execute it guys okay it is saying only tensors of floating point numbers data time can require gradients so this is pretty much important guys previously I have actually defined a floating point number okay so you also have to define a floating point number so I will just write 2.3 dot one dot 4 dot Phi dot 3 dot and again 7 6 4 now once I execute this sorry yes once I execute this now if I just go and see my torch input so you'll be able to see this kind of values now suppose if I try to apply the same equation over here let me just say because always understand this first my inputs will be taken it will be multiplied by the weights then a bias will be added on top of that a sigmoid function will be added and based on the chain rule I also have to clue the backpropagation with respect to all of this particular operation so my first operation basically says that I have to use a torch a torch I have to basically give this input and suppose I am using an operation of square plus 1 suppose I am just using this or okay if I use plus 1 also by the derivative that will become 0 ok so it is fine so let me do one thing let me take ^ - and let me take again torch input to the power of two so this is nothing but it is simple this equation can also be represented as y is equal to X to the power of 3 plus X to the power of 2 which can also be represented like this right X to the power of 3 plus X to the power of 2 now once I execute this right I know what will be my Y value see in each and every variable first of all what we have done initially we had this kind of variable right yes now it is being 12 what how I've actually got this just imagine if I do 2 to the power of 3 what will happen 8 right and then 2 to the power of 2 right it will be 4 so 8 plus 4 will be nothing but 12 similarly if I take this 3 variable and now I am going to perform this particular operation so 3 to the power of 3 is nothing but 9 right plus 3 to the power of 2 sorry 3 to the power of 3 is nothing but 27 and then plus 3 to the power of 2 which is 9 so over here 27 plus 9 is nothing but 36 like this all the operations are actually happened in this particular stage right now what I can do is that I can also apply some some inbuilt function on top of Y because see as I said I've taken the inputs I multiplied by the weights I've added a bias on top of that I can do any kind of operation so here also what I am doing is that and I'll do something like Y dot some ok so if I do y dot summer if I see Z what I'm doing I'm actually adding all these particular variables now ok so after I add it I'm actually getting 1 0 force now this is pretty much important ok whatever we we are getting this right gradient function it basically says that yes we can find out the derivative when we do the back propagation now in order to do the back propagation guys we just have to do something like this ok ok guys now I have got the output of this J Z as a tensor which is nothing but Y dot sum we are basically added all these values and we can see 1 0 4 0 now let me do the back propagation so if in order to do the back propagation I'll just write Z dot backward and I'll execute it once I execute it guys remember what will happen after the backpropagation suppose this is my equation right it will get converted like three X ^ 2 + 2 X because this is getting we have tried to find out the derivative of Phi with respect to X what is this X X is nothing but this torch underscore input that we have initialized from here right so if I really want to find out the gradient with respect to the tensor underscore input that is sorry torch underscore input so what I am going to do is that I am just going to copy this over here I'm going to execute this and I'm going to basically just type dot grad now remember one important thing okay when I did X to the power of 3 plus X to the power of 2 I got 3x to the power of square plus 2 X now suppose if I try to replace this x value suppose over here this 2 is getting replaced so what will happen to the power 2 to the power of 2 is nothing but 4 right so just just imagine this particular number guys over here the chain rule of differentiation is also happening okay so just just see this okay I'm just going to show you over here so suppose if I try to take this first element after the first derivative so this will actually become or just let me write it over here okay this will actually become 3 multiplied by X to the power of or X is nothing but 2 since I am taking the first element ok so 3 multiplied by 2 to the power of 2 right this is what it is happening right and then this is after the derivatives like what this derivative we got I am just trying to compute in based on that then this will be 2 multiplied by 2 right 2 multiplied by 2 because this is 2 X now once I execute this you'll be able to see that I am getting 16 now with respect to this particular term once I do one back propagation I will be getting 16 right I hope you are following it let let me just consider for my second example 3 right so it will be 3 multiplied by X X is nothing but 3 right 2 to the power of 2 that is 3 to the power of 2 okay because we are actually doing this after 1 derivative plus 2 multiplied by 2 instead of 2 I will write it as 3 if I execute it guys I am getting it 3:33 sorry so after one derivative what we are just doing we are finding out the derivative and that particular a back propagation this is just like chain rule of differentiation it will happen with respect to each and every number guys each and every numbers this is pretty much important to understand and this is how a back propagation will actually happen right I hope your understanding is guys see this this is pretty much again I'm going to repeat it initially my equation was like this right after one back propagation I'm just going to compute the derivative so it will be 3 X to the power of 2 plus 2 X this is pretty much simple now if I really want to find out what will be the grad value what will be the this gradient value right how I will be doing I'll just take this number I will insert it over here ok and with respect to each and every number I will be getting the value so for 2 when I try to equate it over here in this particular equation right 3 multiplied by 2 to the power of 2 plus 2 multiplied by 2 I was getting somewhere around 16 in my second case I actually replaced it with 3 right so it became 3 x 3 to the power of 2 plus 2 x 3 then I got 33 right so similarly all the elements will be getting computed now this is how it is basically happening in the neural networks which has many number of inputs many number of weights by using this backward right and always remember we'll be using this functions a lot requires underscore guy grad is equal to true when we will be using this function that basically indicates that yes we have to do the back propagation with respect to this particular torch input and whether it may be any number of dimension of elements always remember the chain rule of differentiation will happen with respect to each and every number that is present over there now this was an example to show you guys how the back propagation using for my torch is actually happening we can set it for any number of Reeboks right any number of red box so please just try it because in the next video I am going to start artificial neural network with PI torch please I would like to suggest you all guys go and check out my complete deep learning playlist to understand the theoretical part in videos I'll be discussing more about how you can implement a and n CN n and RNN with the help of pi touch so yes guys this was all about this particular video I hope you liked it please to subscribe the channel if I'm not already subscribe as CL in the next video have a great day thank you one and all bye bye