Curse of Dimensionality Easily explained| Machine Learning

hello today we'll be discussing about something called as curse of dimensionality this particular topic is a very important topic guys because this is something with respect to the understanding of feature selection and feature engineering the curse of dimensionality so let us just understand what exactly this so when I caused say something about dimensionality this is nothing but dimensions no this is something called as dimensions dimensions are also called as features features it may be independent feature or it may be a target output feature features are basically like attributes it is also called as attributes okay it is also called as an attributes now let us just consider some example I'm going to take a data set and in that particular data set suppose I want to create some independent models the only condition that I am going to change is that in this particular model from that particular data set I am going to take new features from in the next model I'm going to take five features in my model M 3 suppose I am going to take ten features and then in m4 I am going to take hundred features when I say I'm taking hundred features that basically means I am considering the hundred independent features from there and the target output I am actually trying to create a model or tree in the model on that particular data similarly if I take in my m5 model like two hundred features and over here I'm going to take ten thousand features and here I'm going to take ten thousand now let us just consider a very simple example suppose I consider that I'm to find out what is the price of the house with respect to the size of the house and number of bedrooms okay if I take this scenario I have two independent features right and I have one output feature that is my price I'm going to give this or I'll go to Train this particular model M 1 by using these two features and one my output features which is called as price now in my next model what I do is that and suppose I get some accuracy EC 1 I'll say a CC 1 ok now in my next model which is my defendant model m2 I'm going to give the same dataset but instead of just giving two independent features I am going to consider five independent features now when I say five independent features it may be something like state okay it may be something like what is the bedroom size okay any number of features that I can additionally add you can just think of that you are just trying to make up some features than you are trying to provide this to the model right now when I give this five features to my model m2 and I'm a tree in it it definitely gives me a better accuracy than the previous one right because in this particular model I'm going to give a data set which has enough information for the model to learn more about the data so the accuracy one that we caught it over here will be always less less than accuracy to which is actually found out by the model two similarly when I go ahead and I try to create my next independent model that is m3 and but here the condition is that I am going to take ten independent features from this particular dataset okay and it's a time independent feature I am going to pick that ten features along with the output okay now this particular model M three will also give us some accuracy which will be like my AC C three and suppose this particular model also gives me a very good accuracy and this accuracy is greater than M two and it is greater than M one okay now similarly as we go ahead okay as we go ahead by increasing the features our accuracy mean is increasing right but after some time after a threshold value suppose my threshold value where is ten features after this particular threshold value what usually happens is that as we increase the number of features right our model will not be giving much more accuracy than the previous one but as we explained from here as I am going from M 1 to M 3 I'm increasing the number of features and we saw over here that our cracy was actually increasing but when we went to M 4 you know after reaching a threshold value of the number of features B we are actually giving away 100 features right and we are trying to Train that particular number of features in mind next independent model that is M 4 and Here I am finding it out that the accuracy that I am getting over here which is my ACC foe will be less than this particular empty right now here is what we say with respect to the curse of dimensionality right it is not always necessary that as you go on increasing the number of features your model will be able to give you a very better accuracy than this one okay and similarly now after this what will happen if I have this ACC for right then similarly I will create an independent model m5 and here I will be giving my two hundred features from the data set and over here you will be seeing that you will be getting an on the accuracy ACC Phi which will be worser than this particular ACC form you know because the number of dimension is still more increasing now why does this particular thing happen and similarly it happens with respect to one hundred feet thousand features and ten thousand features as you increase the number of features after the threshold value your accuracy actually degrees it but the next important question is that why does it actually happen right that you need to understand now see as we are going from m1 to m2 and m2 two and three as we are increasing this number of features still the threshold value the model is able to learn more and more information from that particular data more and more unique information more and more use necessary information to actually predict the price or the target output of the particular use case but after the threshold value what happens is that as the number of features increases exponentially like this is the most important term if it increases exponentially the model gets confused because you are feeding him a lot of information from that particular use guest cells so the model gets confused it literally you know the thought of that particular model literally scatters out it will not be able to observe all that particular information hence the accuracy decreases you know so the confusion on that particular model after learning with respect to so many number of features it tends to make the accuracy of that particular model lower and similarly after this particular threshold point when we see that our model how features is actually increasing exponentially from 100 to 200 mm to 10 decreasing decreasing and that is what is called as cursive dimensionally now this is very important guys because in my next video I am going to discuss about them how we should select like how many number of features I have to use forever for a particular problem statement whether I should be using ten twelve right from that particular number of features so there are a lot of tests you know there are some tests called as choice square test there is something called as correlation coefficient okay all these features will be actually necessary to actually find out like how many number of features will be used so in my next video I will be coming up with the feature selection part and then which will also be a very important part of the feature engineering I hope you like this particular video guys please do subscribe to my channel if you have not subscribed and please share with all your friends like comment I'll see you all in the next video have a great day head thank you one and all