Machine Learning Foundations: Ep #7 - Image augmentation and overfitting

Key Takeaways

hi and welcome back to machine learning foundations I'm Laurence Moroney your host and guide and I'm here to step you through the basics of machine learning from a coders perspective over the last few videos you have looked at convolutional neural networks and how they can be used for computer vision you build classifiers for fashion horses and humans and cats and dogs but one theme that was common was the concept of overfitting were given limited data to train on your neural network could get very good at recognizing this data that it was trained on but not necessarily very good at recognizing any data that it had not previously seen to help this make sense consider this scenario say your whole life you had only ever seen shoes that look like these then to you that's what shoes would look like their cover the ankle that have flat soles that have laces all those kind of things so then if I showed you this picture you'd instantly recognize everything here is a shoe the big ones the small ones the ones on their sides they're still in your mind shoes but what if I show you this picture now we know this is a shoe because we've been exposed to shoes that look like this but again if all you had ever seen is the shoes I mentioned earlier you wouldn't recognize this there's no laces the sole is all wrong and it probably doesn't cover your ankle you would then be over fit in the data that you think makes what a shoe is the same happens with a neural network if it hasn't been exposed to a label sample that looks like a high-heeled shoe then it's the same as our hypothetical person who has only ever seen ankle boots as shoes of course if you don't have the data you don't have the data but there are many cases were neural network overfits because of attributes in the image that it may not have previously seen but using something called image augmentation we might be able to fix this this might seem a little vague so let's show an example on the left is a cat and on the right is a person computer vision works by isolating features so you might have features like the pointy ears at the top being indicative of a cat or the two legs at the bottom being indicative of a human but in the case of the cat for example the image on the right is a cat you and I know that but a computer may not because if it was only trained on images like the one on the left then it's looking for triangular shapes oriented upwards near the top of the image but the ears on the cat on the right don't look like that so what if on our training set we could rotate the images so we train with a labelled image of a cat that looks like this then the ears are the same as the cat on the right so by using a transform like a rotation we're effectively creating new data to be able to train on the good news is you have all the tools you need all ready to get started on this using image augmentation to artificially extend your data sets to provide new information for the training of your neural network and this can really help you with overfitting issues to get started you can use the image data generator you've already seen this for image manipulation with the rescaling that you've been doing to normalize the image and it supports more parameters so let's explore some examples so here's an extended image data generator constructor where I've added a bunch more parameters as well as the rescale one that you've already seen you've already seen rotation range this will randomly rotate each image by up to the parameterised amount plus or minus so here you can see it's 40 degrees so the image will be rotated left or right by that amount width and height shift range will move your image around in the frame by that amount so here it can be moved by up to 20 percent in width or height which can give an effect like this where the image is width shifted and the subject moves to the left sheering can give an excellent effect and he set it up with this parameter the value is from 0 to 1 so would point to like this you can shear by up to 20% to see where it could be useful consider these images the image on the left is from the horses or humans data set and the person is standing with his hands raised the image on the right has nothing like it in the data set and as such it might fail to be classified a human were overfitting to standing poses for example but if the left image was sheared it can suddenly look a lot more like the image on the right the network while being trained could be trained to recognize an image like the one on the right and would be no longer overfit just for standing poses another example is the zoom range and this parameter gives us a value from 0 to 1 to zoom by this will give us a random zoom of 0 to 20% of the image size again let's see an example of why this is useful the image on the left here is a woman from the training set from horses or humans the woman on the right is quite similar but because of how the image is framed we can't see her legs and again the neural network might be over fit to seeing full bodies that include legs the lady on the right is obviously human but the neural network may not recognize that but if we were to zoom in on the image on the left it now has far more similarities to the woman on the right and we may be able to recognize the woman on the right as a human if we train with data like that on the left another example is horizontal flip if this is set to true then images will be randomly flipped and let's take a look at an example of how this could be effective the image on the left is from the horses and humans data set as you can see she has her right hand up the woman on the right is quite similar but she has her left hand raised if the training set only has images like the one on the left we could over fit for humans with their left hand raised but if we randomly flip we could effectively extend our data set to include left hand raisers film mode equals nearest is used to fill in parts of the image that could be lost in some operations like skewing so that's a tour of some of the operations that are available with image augmentation a really cool tool to help you avoid overfitting in image based data sets but let's put it into action next I'm going to show you a screencast of exploring image augmentation with a reduced version of the cats and dogs data set that has only 2000 images and we'll see the impact so here's the cats versus dogs example and what I'm gonna do is use a filtered version of the data set that you can download from here that only has two thousand images because it has less images it's going to be more prone to overfitting we'll take a look at the impact of using image augmentation on that overfitting but first of all I'm going to create one that does not use image augmentation so I'm just going to train a neural network with it I'm going to use exactly the same parameters later on one that does use image augmentation so we can see how it all works so you can see here it's downloaded the image is setting everything up and it's going to start training the model it's going to train for a hundred epochs we can see the training is beginning now and then I'll speed it up and see how it works later okay now we can see that it is finished training and the accuracy on the training set was 99.85% so very very high but on the validation set it's a bit lower at seventy-five percent so that's clearly showing some kind of overfitting and particularly we're expecting that because we're using a smaller data set if I plotted out the training versus validation accuracy we can see that it actually flattened out at around about thirty epochs and the validation accuracy flattened out at around the same but you can see are lost drastically increasing on the validation set clearly again showing that we're overfitting terribly so what can we do about this so one of the things that we spoke about in the video was the fact that we can do image augmentation so here's an example of using the image data generator rotation range up to 40 degrees in either direction width and height shifting 20% in either direction shearing 20% zooming 20% and then randomly horizontally flipping the image just like we had seen earlier so let's now rerun the code and this code is exactly the same with exactly the same model except in this case we're going to run the image data generator code to do the image augmentation so if I start running this it's downloaded the cats and dogs it's giving us two thousand images in two classes and a thousand validation images and it will start training one thing you'll notice is that the training is a little bit slower this time because it's doing the image augmentation on the thousands of images so while earlier if I go back we'll see our epochs were taking about nine seconds we'll see this time round even though we have a hundred epochs it's the same it's gonna be a little bit slower it's taking about 19 seconds so we've about doubled the training time a little bit over double the training time because of the fact that we're doing all of that image augmentation so now as we can see we finished training we ended up with 85% accuracy on the training set and 83 percent accuracy on the validation set now while this may not look as good as your 99 plus percent that you had earlier on at the point is here that is that it's not overfitting we have 85 percent on training and 83 percent on validation data that it hadn't previously seen so this model realistically is 85% accurate at predicting cats versus dogs and when we look at our training and validation accuracy we can see the curves are fit together very nicely and the training and validation loss are also fit together quite nicely maybe here in some of the later epochs they're beginning to diverge so that it's showing that's beginning to overfit at that point but we've probably over trained the network at that point we might want only want to go for about 60 epochs or something like that so that we can be sure but again this looks very very nice now with the accuracy and validation being very close to each other and we got that by using the image augmentation another little trick for avoiding overfitting is to add drop out now we didn't cover that in the video so I'll just show a little sample of it here where the only thing that I've done differently is I'm adding this drop out layer to the model after everything and I'm gonna be pretty aggressive and dropping out and that's dropping out half of the neurons so in this case it's to prevent what's called a proximity bias when neurons that are close to each other can be trained with very similar values if you drop out a bunch of them you can get rid of that and again you can prevent overfitting so let's run this one and it's gonna do the same veinte ssin that we had earlier on it's downloading the data it's setting it up it's gonna start training it the training here may be a little bit slower let's take a look well again it was the same it was about 19 seconds so they drop out wasn't impacting it in any way but let's train this now for a hundred epochs and then after 100 epochs we'll take a look at what the validation and versus accuracy curves look like now that we can see we finished recording we're at about 82 percent accuracy and about a little over 82 almost 83 percent accuracy on the validation sets we might actually even be underfitting slightly but again the getting accuracy and validation accuracy as close as possible to each other as a sign of a really strong model and you can see we have that here we're getting some wild swings here in the later epochs and that's probably because of the drop out and we're training a little bit too long but again using dropout or at getting our accuracy in a validation accuracy closer to each other to avoid that kind of overfitting sometimes you can go too far and maybe under fit slightly not sure if that's the case here because if we take a look at previous epochs it's usually the training accuracy is a little bit higher than the validation one so that's it those are some techniques particularly augmentation that will help you avoid overfitting and I hope this has been useful for you here's the URL pause the video and give it a try for yourself and that's it for this episode in the next one we're going to switch gears and start looking at natural language processing beginning with the concept of tokenization I'll see you then and whatever you do don't forget to hit that subscribe button [Music]