Simple MNIST Convnet - Keras Code Examples

Key Takeaways

welcome to the henry ai labs walkthrough of keras code examples keras has provided 56 code examples implementing popular ideas in deep learning this ranges from the basics such as simple mnist and imdb text classification all the way to cutting-edge research ideas such as knowledge distillation supervised contrastive learning and transformers we'll also explore fun generative examples like variational autoencoders and cyclegan my contribution to these code examples is to explain every single line of code in each of them walking through each of the individual keras examples i'm not the author of these code examples please consider starting the github repositories to show support to the original authors from this video you'll learn how to load the mnist dataset using the keras datasets library how to preprocess this dataset for a quick input into a simple neural network and then how to train the model and evaluate it quickly with this test set that's been loaded into memory with this dataset library so as stated in the title this example is a bit easier simple mnist with a convolutional network so first we import numpy keras and keras layers so now we've prepared the data so here's the first interesting thing we've seen in our series keras has built-in data sets torch vision torch text and keras they all have some built-in data sets that you can load data sets just by calling this api call so you do keras.datasets.mnis.loaddata so this is much different from in the last example we had to get our data by doing a curl and then grabbing this link from the web then we had to pipeline it from the disk into our workspace whereas keras has some examples of data sets that are built in that we can just do datasets.mnis.loaddata so mostly this is just academic datasets that are accessible like this but it's definitely a trend in this direction which is really exciting hugging face they recently did an nlp library where you have this kind of easy syntax to load data into your models so another interesting thing about this code snippet we see we have the 10 eminence digits the input shape here's the other interesting thing so when we use this api to load our images they're originally stored as 28 by 28 dimension so this is describing the height and the width of the mnist digit represented as a grayscale image so in order to make this compatible for our deep neural network we're going to do numpy.expand dimensions x train -1 so as shown in this code that i've added what you do is if you have a numpy array that's 28 by 28 and then you expand the dimensions you add this one channel in the back just to make it describe the 28 height 28 width one channels is in grayscale so the other thing we do is we do keras.utils.2 categorical to convert our y train from say nine or six into a one hot encoded vector for the class label so now y train is gonna describe say it was a two it would now be zero zero one zero zero zero zero this is how we do the categorical cross entropy loss when we're fitting these labels with our machine learning model so now we're building the model compared to the first example we build this complex exception network here we're just using two convolutional layers with 32 filters 3x3 convolutions interleaved with max pooling then we flatten this feature plane apply dropout with a 50 probability and then apply a dense fully connected layer from that flattened feature vector into the 10 output classes so then we also have model.summary so this is model.summary from keras and it is really useful because it shows us the intermediate dimensions of the feature planes in our deep neural networks so we start off with the 28 by 28 mnist digits we pass it through a three by three convolution that down samples the spatial resolution to 26 by 26 note how this is two less than the original height and width of the mnist digits we have 32 such feature planes so the dimension of this intermediate tensor in our deep neural network is 26 by 26 by 32 and we have 320 parameters in these convolutional filters these are the parameters that we use to transform these intermediate feature planes in deep neural networks now we apply max pooling 2d this is going to be a spatial pooling operation that's going to take the max value between every two pixel slots and slide that across to reduce the spatial resolution from 26 to 13. then on this 13 by 13 by 32 feature plane we apply another convolutional 2d that down samples it to 11 by 11 and this convolutional layer has 18 496 parameters then we do another max pooling and then we flatten this out into a vector so we take these five by five we have 64 five by five feature planes and then we flatten that out into a one dimensional vector that has dimensionality one thousand six hundred by one because it's a vector then we apply dropout and we match from one thousand six hundred into the ten output classes so all in all the simple neural network has thirty four thousand eight hundred and twenty six parameters most of which are contained in the second convolutional layer and the output mapping from the dense feature vector into the output classes another really important theme in our series walking through keras code examples is how much faster models run on gpus compared to cpus so this is running the model this simple convolutional network to fit mnist on a cpu note how it takes about 33 seconds to step through the data set now that we've changed the runtime to the gpu provided by google collab it takes one second to step through the data compared to the 33 seconds when using the cpu now that we have a trained model we can use the short syntax provided by keras to evaluate our model with our previously loaded test set so remember that we have this api where we load in the x text and y test we normalize x test by dividing each value by 255 expanding the dimensions of each image so it's compatible and then changing the labels to be categorically one hot encoded vectors compared to just say a numeric number say three four or five that would denote each individual handwritten digit so we can use this quick syntax model to evaluate our data sets that are already loaded into memory and then from this we get the score and score at index zero is the loss and score at index one is the accuracy on the test set so we quickly achieve 99 accuracy on mnist using this simple model because mnist is a is a toy example and it's easy to get a good model with this so to recap some of the big takeaways are noting how we have these data sets that are loaded into keras these are mostly academic data sets like mnist or cfart10 or maybe stl10 these kinds of academic data sets are loaded in computer vision as well as text data sets like imdb and maybe some semantic text similarity data sets so we also note this expanding dimensions converting labels to categorical a quick sense of how we can build convolutional networks and just more experience and familiarity with this syntax training the model noting how much faster it is on the gpu and then a shorthand syntax to evaluate a model on a test set that's already been loaded into memory