Bias and Variance for Machine Learning | Deep Learning

bias and variants are two of the most important topics when it comes to data science they are this important because they lie at the base of many critical concepts like overfitting and underfitting and they also tell us some ways of how we can deal with overfitting and underfitting especially for a beginner data scientist or for someone who is just learning the ropes of data science this can be a little bit tricky to learn because people generally tend to try to learn it by heart try to understand okay high bias and high variance means this or low bias and low variance means that but actually there is some really solid logic behind it and once you learn that logic everything becomes much easier so that's what we're going to learn in this video we will learn what bias and variance is in a solid way and then we will see how to deal with high bias and high variance this video is brought to you by assembly ai and it's part of our deep learning explained series assembly ai is a company that is making a state-of-the-art speech to text api if you want to try assembly ai yourself you can go grab a free api token using the link in the description all right let's start with bias so bias is basically the amount of prejudices or assumptions your model is making against a certain problem that you're trying to frame the more assumptions your model has the higher the bias is going to be on your model one model with a very high bias is linear regression the reason linear regression has high bias is because it assumes that the data has a linear distribution and what happens when you give it data that is not linear is this you might not be able to fit the data at the end because of all the assumptions that you're making about the problem or the data and what does this look like you might have seen before it looks like underfitting so that's why high bias so a lot of assumptions means underfitting variance on the other hand shows us the sensitivity of your model on the training data so it tells us how much the output would have changed if we change the training data even in the same problem when the model is dependent on the subset of the real world that you're training it on what happens is if you change that subset in the real world that you're selecting to train it the outcome is going to change dramatically a model like this will not be able to perform well in the real world so what you have at the end is a model that is overfit to the training data so as you can understand high variance means overfitting models that have high flexibility tend to have high variance like decision trees so if you imagine a decision tree if you do not put any limitations on this tree it will grow and branch up as far as possible to fit all and every single data point as possible while training a model like this that has fit every single data point would give us really good results but in the real world it will fail to perform with traditional machine learning algorithms there is always a little war between lowering bias versus lowering variance because when you try to lower one the other might shoot up and you want to strike a balance there you're always trying to fit the data well enough so that you're not under fitting but at the same time not over fit the training data so your performance in the real world is still protected this is called the bias and variance trade-off but with the latest development in machine learning especially with deep learning we don't really have to worry about that anymore because we have some tools in our hands that will lower bias only or lower variance only so let's talk about some of those solutions if you have high bias as we said what that means is that you're under fitting and one of the best things and first things you can try is to train your model more because maybe you are just you have just not trained your model enough and your model did not have yet time to converge to a good solution and that's why you're under fitting your problem so trying to train it a little bit longer might help you solve underfitting next thing you can do is to try increasing the complexity of your model because maybe your model has way too many assumptions and it's way too simple for the data that you're trying to frame so this might look like if you have a decision tree with maximum depth set to two increasing that to five or ten that will create that will make your model a bit more complex and at the end it will lower your bias another thing you can do is to change the model architecture that you're using maybe you are using a architecture that does not fit your problem really well so for example maybe you're doing image classification with deep neural networks but maybe trying a convolutional neural network architecture would work better for you and you might not under fit at the end next we have high variance and what kind of model do we have when we have high variance it is a model that has overfit one of the first things you can do is to try to train it with more data if you can't this might not be possible always because sometimes getting more data is basically too costly or not even possible but if you're overfitting introducing more data to your training if possible is always a good idea another thing you can do is to use regularization on your model so what is regularization we talked about this in a previous video and i will leave the link to it somewhere here but basically regularization what it does is lower the complexity of your model so you can see it's basically doing the opposite of what we did when we were trying to deal with high bias by using regularization you will be limiting the flexibility of your model or the complexity of your model basically decreasing the complexity of your model and will be able to lower the variance in your model and lastly again you can try a different model architecture maybe the model that you're trying to use right now is just not a good fit for the problem that you're trying to frame so trying out a different model architecture might help you combat overfitting as i mentioned now there are ways to lower bias without increasing variance and vice versa so one way to do that is if you're under fitting you can increase the complexity of your model and use regularization on top to avoid overfitting that comes from the high variance of increasing the complexity of your model if you're overfitting on the other hand if you introduce more data to your training you will be lowering the variance without increasing the bias of your model and that's it that's all you need to know about bias and variance let's do a quick summary bias is the number of assumptions that your model has if it has too many assumptions it will have high bias and that would lead to underfitting variance is the sensitivity of your model to the data that it is being trained on if it has high sensitivity that means you have high variance and that means that you're overfitting to deal with high bias what you can do is to train your model more or increase the complexity of your model whereas to deal with high variance what you need to do is to decrease the complexity of your model or introduce more data to your training if you understand all of this that means you have a good understanding of bias and variance from now on it will be easier for you to understand what to do when your model is under fitting or overfitting thanks for watching and i hope you enjoyed this video if you liked it don't forget to give us a like and maybe even subscribe we would also love to hear about your questions or comments in the comment section below but before you go away don't forget to go grab your free api token from assembly ai using the link in the description have a nice day