Pytorch LeNet implementation from scratch

Key Takeaways

[Music] in this video I want to go through an old-school convolutional neural network architecture and that comes from a paper by Yonghwa Kuhn and the network is called Lynette the Lynette architecture looks like this and it starts with a 32 by 32 image input and in the in the first layer it uses a 5x5 kernel with a stride of 1 padding of 0 and gets it to 28 by 28 it also uses a our channels of this two v6 and then in the next layer the Linnet architectures is something a little bit unique but essentially it's an average pool layer with a kernel size of 2 and a stride of two essentially having the input size so 28 becomes 14 and then in the next layer it uses the same 5x5 kernel and sraight of one padding of 0 to get it to 10 by 10 and also uses the out channels of of that layer to be 16 then in the next it uses the average pooling layer again to have the input and then it uses the 5x5 kernel again so it uses a 5x5 kernel with a stride of 1 padding of 0 and gets it to 100 and the output channels is 120 so this is a convolution air layer with 120 channels 1 and one by one then it uses a fully connected layer linear layer with 120 to 84 nodes and then again a linear layer with 84 to the 10 outputs I've summarized the Linnet architecture here so we use a 32 by 32 input and this might be a little confusing since the Linnet architecture was was built for the amnesty to set but the Emnes data set has 28 by 28 and I'll reference to the paper but essentially this was improve believed to improve the performance yeah I'm not sure if this is actually true because modern architectures don't use this but that was the reasoning behind it and then we just used a 5x5 kernel average pool 5x5 average pool and another comm 220 channels we flattened it we use a linear so 120 to 84 and then another linear time for the output which is 10 so this is just the reference we can use so let's actually try to build this so we're going to use class Lynnette and we're going to inherit from the NN module to find the innit the first thing we always have to do is call super of vennett and initialize the parent method which is the end of module and one thing is that so Linnet uses 10h and sigmoid activation functions we're gonna use relevance Ted because I mean there's no reason not to use really the only reason was that it wasn't invented at that time and really the goal here is to understand the architecture not to implement every single detail as laocoon did so we're gonna use another weight initialization as well but really yeah so the goal is to understand the the general architecture so we're gonna use self dot relu to be and then dot braille ooh we're going to create the pool layer that we're going to use which is an average pooling and the kernel size will be 2x2 stride will be two by two and essentially so the average pulling is what we use here and use here we can reuse the same pooling because the the average point doesn't have any parameters and then let's initialize the comm layers so the first account layer is going to be so cnn.com 2d with an in channel of one our channels will be six kernel size will be 5x5 stride 1 padding:0 I don't think this is actually necessary to write but just to make everything clear we write it and then com2 we really going to use exactly the same the only thing that's going to differ is the number of in channel since this is the second comm layer we need to match the our channels of this to the in channels here so this is going to be 6 and lacunae uses 16 as our channels here and then it uses another comp with the exact same kernel and stride padding except we need to match again 16 and the out channels here is going to be honored in 20 then we need to have the linear layers so and on top linear 120 84 and then the linear 2 and then that linear of 84 and out for the number of classes which is 10 then we want to do is that we won't define the forward first run it through the comm Blair the first calm then we use the relative activation function and we just call the output of this X and then we're gonna use our the pooling then we're gonna use so basically the same again except we're gonna run it through chef com2 and reuse the pooling and lastly we're going to use again this except com3 then what we want to do is that we want to so the output of this will be yeah so the number of examples that we have by 120 channels by one by one we want to make this to the number of examples by 120 essentially we just want to make these into a single dimension so self dot yeah so X dot reshape and we want to keep the first dimension the same the number of examples and then just minus one then now we can run it through the the linear so X set the linear one of X and just call this X and then use the rel loop now for the last output yeah so for the last output what we're gonna use is just a linear the linear to no activation function on the output layer so and then we're just going to return X and that's essentially the architecture so let's see if we get it to run we do this by creating some just random input and then calling the Lynette and yeah so we can do print model of x-type so what we expect this to be is 64 by 10 because we have 64 images and for each we want to have like the prediction for each of the integer values yeah so if we run this damn it okay so yeah so here this should be calm three yeah great so then we get 64 by 10 now if you want to use this for the M list all you have to do you can follow my last video I did which you uses the Eman estate asset and all you have to do is just pad the images by two and then you can just run it through this architecture if you have any questions then leave them in the comment thank you so much for watching the video and yeah I hope to see you in the next one