3. Convolutional Neural Networks

Key Takeaways

we need to talk about convolutional neural networks so far everything that we've done applies to any type of input multi-layer perceptrons is a generic algorithm that you can use on literally any kind of input if you can get it into an array of numbers but convolutional neural networks only apply to certain data sets and images are one of them the reason is when we apply a multi-layer perceptron it throws out a lot of data when my little friend here looks at something and gets an image the image actually has some structure to it it's a 2d array and the XY coordinates we have an intuition that they do matter and if I took an image and I shifted every pixel to the left all the numbers change but it's actually in that image probably stays the same so we need some new mathematical operators that take advantage of that and the operator that's generally used is called a convolution convolutions have been around for a really long time even convolutional neural networks have been around for a long time but around 2011 or 2012 applying convolutions to object recognition inside images with neural nets went mainstream with huge success the task of recognizing what's in an arbitrary image went from being basically impossible for any machine learning algorithm to having better than human level accuracy in many cases using CNN's this kind of improvement for mere 30 percent error rate to better than human accuracy over just a few years is probably one of the biggest breakthroughs in AI and it's probably a big part of the reason you're actually watching this video let's apply convolutions to our image recognition problem what is a convolution now generally speaking there's one D convolutions and two D convolutions and three D convolutions and you might have seen a convolution in signal processing or in your math class involving somehow Fourier transforms but in this case what we mean by convolution is actually a 2d discrete convolution and so you can think of that as sliding a window of weights across an image or more practically you can think of it as very low-budget Photoshop filter so assuming you're in the right directory you can find a little tool I wrote called con demo this is mostly for demonstration purposes I start with the kernel set to a three by three array where each value is equal to 0.1 and now what this convolution does is it scans across the image and it takes each pixel value in a 3x3 chunk and it multiplies it by 0.1 and then sums the number and then puts the sum in the first value of the output image now we step one pixel of the right and we do the same thing with the same weights and we do it over and over and over for every pixel in the image this convolution basically averages over 3x3 block so essentially you can think of it as a blur okay so what do you think this does if I make the middle of the kernel 1 and all the other numbers 0 I actually get the original image back and now what do you think this does if I make the middle of the kernel 2 and all the other numbers 0 it actually makes everything brighter which kind of makes sense because it multiplies each of the pixels by 2 exactly and now I won't go through this I'll leave this to you but you should think about what does this do if I make the middle of the kernel 1 and the number of above it negative 1 or what do you think this does if I make the middle of the kernel 1 and all the others are numbers around it negative 0.1 and if you play with this conv demo for a while you'll get a great intuition for what convolutions can do there's actually one more piece of this that's important which is how we do convolution on multiple images and a really important case is where this happens on color images so actually my little dog was originally a color image so the way we pass it into the neural net is we broke it up into red green and blue components so actually we do a 3x3 convolution on the red data the green data the blue data and then we add up all the results so sometimes convolutions take a step of more than one if each iteration moved by a step of two we would say that the convolution had a stride of two how we handled the edges also matters if we do a 3x3 convolution on an image and we don't go over the edges the output image is a little smaller than the input image this is what Kerris does by default so if you want to preserve the image size you have to add zeros around the input image and that's called zero padding now there's another simpler transformation also very common in neural networks called pooling if convolutions are a bad photoshop filter pooling is like a bad resizing algorithm typically pooling takes a 2x2 region of an image and chooses the max value in each region this is max pooling average pooling takes the average in each region either way this shrinks the image by a factor of two you can actually achieve similar results with strides but we want to shrink down the image so we can do convolutions at different scales now let's go to the code and put it all together so open up CNN Pi and you'll notice that this is very very similar Tori left off in the last video of a multi-layer perceptron but there's a nasty little wrinkle on lines 19 and 20 and that's because Kerris is 2d convolutions want three-dimensional input because most images are actually two-dimensional but then have a third dimension for color our images happen to be two-dimensional only because they're grayscale and they don't have a color dimension so we need to use the handy reshape function that takes our image and adds an extra dimension of length one to fit Harris's convolution API now line 29 is the others line that changes until now we've actually flattened out all of our models for perceptrons but in this case we're going to do a convolution instead of the flattening notice that we still have to explicitly tell our model that the input shape is 28 by 28 by 1 we're actually doing 30 to 3x3 convolutions in parallel and we're going to need to learn the weights of each of those convolution parameters next we had a max pooling layer which shrinks down the network then we had a flatten layer because the next and Slayer is a perceptron and expects a one dimensional input let's try running our model but before we do it let's be sure to call model death summary and check out the output okay because we're in a new directory we need these called W and B and knit one more time to set this up we can use the same user name and we can optionally select a new project or use our previous project if you've been following along with previous videos now Python CNN PI actually trains the model and here's where a GPU would start to really speed things up there's only 320 parameters in our convolution layer but they dramatically affect the output you can see because we don't have pooling it actually shrinks our image input from 28 by 28 to 26 by 26 in the convolution but it actually makes 32 different output images so the output size of our convolution layer is really large the max pooling layer has no free parameters because it always does the same thing but it cuts the size of our images in half the flattened layer of course also has no free parameters the hidden dense layer has a ton of free parameters because it has a hundred and twenty eight outputs and five thousand four hundred and eight inputs our total model has nearly 1 million free parameters and only 60,000 training data points so what should we be worried about I hope you guessed overfitting because if you did that means you've been paying attention and I hope that you're thinking if I have overfitting I should do one thing and that is to add dropout now if you were my student and we were in person I would make you do that on your own to check understanding I can't do that on the video but I think you should try I'll show you how to add dropout to the network and there's more than one way to do it here's how I would do it typically you want to put dropout between layers that actually have a free parameter so in this case it'll be after the max pooling layer and then after the final dense layer you could also put dropout before the very first layer typical inputs to the dropout is about 0.4 which means that 40% of the things coming into the drop out gets set to zero you could set this anywhere between 20% and 50% it tends to not matter that much this is gonna take a while to train so what we wait I'll tell you about one more improvement that we have we're actually at 98% accuracy so but to get to 99% accuracy we need to have multiple convolutional layers and the intuition for why is it a convolution only acts at one scale but if we shrink the image down and we do another convolution then we can detect patterns and multiple scales so a typical architecture for most object recognition tasks will actually have multiple convolutions and some kind of shrinking operation in between the convolutions max pooling is probably the most common but you'll see other strategies for shrinking the image down so this diagram actually lays out a very typical convolutional neural network the bottom here is the input eight and that's actually the digit 8 from the same data set we're looking at and then the next layer from the bottom is the first convolutional layer then there's another max pooling layer followed by another convolutional layer followed by another max pooling layer followed by some hidden layers and if you look closely you can see the output that the eighth neuron is lit up on the top I love this visualization because you can see all the crazy things the convolutions do now here's a challenge can you actually create this network by modifying CNN pi so stop this video and give it a try if you like I do it by adding two layers in the middle of this network you can actually tune the number of outputs of this layer and the number of nodes in the hidden layer experiment see if you can build interesting network architectures that get higher accuracy we're actually getting almost perfect accuracy on this data set at this point so it might be time to try applying this to something else and for your educational entertainment I have a completely new but very similar data set called fashion M NIST it's 60,000 images but instead of being handwritten numbers they're images of clothes and the categories are a t-shirt trouser pullover dress coat sandal shirts sneaker bag and ankle boot in fact if you open up fashion dot pie I have some skeleton code that starts off where this whole set of lessons began with a very very basic perceptron can you apply what you've learned to build a fashion classifier on a similar data set let me know how it goes and comment below okay so so far we've covered the basic introductory class into neural networks we've done perceptrons multi-layer perceptrons and convolutional neural networks this should be a basic understanding that lets you actually use real deep learning code in the wild but from here on out it gets super fun we can apply this to lots of different applications and I want to show you about autoencoders which are a really cool architecture I want to show you about transfer learning I want to show you about text and lots of other things but you should actually enjoy the knowledge you have right now on convolutional neural networks go out and build an object classifier on a new set of data I think you'll really have fun ok yeah you know just came from board meeting and they told me if I don't get a million subscribers next week they're gonna fire me so really mean a lot if you guys could subscribe and like and click on my patreon what else [Laughter]