Custom Ensemble Machine Learning Algorithms

Key Takeaways

hello all my name is krishnan welcome to my youtube channel so guys today in this particular video we are going to see custom and simple machine learning algorithms now if you know about ensembl techniques we have some of the techniques like random forest right this is basically a bagging technique right and then we also have xgboost okay xgboost this is basically a boosting technique now in all these particular techniques you know that we use decision trees and we will be using many decision trees so we basically call it as an ensemble right in boosting also we use decision trees right and then what happens internally the voting classifier whichever gives the maximum number of votes gives the output similarly in the third technique what we see is that you have something called ada boost or you have techniques like gradient boost and many more right so gradient boost and many more you have cat boost right so these all are kind of ensemble techniques which uses only decision trees now what if i want to create a custom ensemble technique so as i said over here i'm going to create a custom ensemble technique and in the custom custom example technique you know what i will be doing is that i will try to create this ensemble with different different algorithms probably i may take logistic regression i may take the second one like decision tree i may take the third example like xgboost or random forest right i may take the combination of all these particular algorithms and from this on top of this after combining all the result what i'm going to do is that i'm going to take the voting classifier voting classifier and whichever gives the best voting maximum number of votings like whether it belongs to class 0 or class 1 i may select that specific output so that is what we are going to do in this particular video okay we are going to create a custom and symbol techniques internally this custom ensemble techniques can use logistic regression decision tree random forest and all the other kind of algorithms and out of it which will be giving the maximum voting classifier we will take that as an output so for this particular problem statement first of all what i'm going to do is that i'm going to create some synthetic data points now synthetic data points are just like artificial made data points you can also take any kind of data set that you want to solve this particular problem and for creating a synthetic data points what i'm going to do is that i'm going to use from scalon.datasets import make underscore classification then inside make underscore classification i will be taking how many number of samples how many number of features regarding the n underscore informative and underscore redundant and with random state so as soon as i create this you'll be able to see that i'll be able to create a data set with 20 features along with this one right so in short this is nothing but a classification problem statement so we will be seeing that as soon as i create it right as i create it and if i use a suppose i want to use uh just let me write from collection okay import counter okay so i'm going to import counter and this counter i'm just going to check it for my y to just understand that how many number of zeros and ones are actually present okay so i'll use counter is equal to this and if i go and see in my next code it will show me what is the total number of counter so here in the counter you will be able to see that the number of zeros is nothing but five zero one the number of ones is nothing but 499. so this is the way how you can actually check how many number of zeros and ones are actually present inside y because if i see over here the x and y will be shown in the form of array okay so it will be for shown in the form of array every record every row will be having 20 features because that is what an underscore 20 features actually uh tell us okay and underscore redundant is something related to redundant if you really want to see the properties just press over your shift tab and you'll be able to see all the features over here each and every feature explanation is given over here just have a look on to it in this case c and underscore redundant the number of redundant features these features are generated as random linear combination of the informative features right so all the other information you can see over here now after i create this what i'm going to do is that in this particular problem statement i'm going to use decision tree i'm going to use random forest i'm going to use as svc okay and then i'm going to use cross val score repeated stratified k folds this is basically for doing some kind of you know with respect to the train test split if i don't want to do in this particular process i can basically use a repeated stratified k fold then after this all these particular algorithms will be combined together whichever will be giving the maximum number of votes with the help of voting classifier we will be able to find out the output and for this we will be creating a pipeline okay so first of all what i am going to do is that i am going to create a variable which is called as models which will be in the list format and then the first thing that i am going to create is decision tree here i may also perform normalization and standardization but i don't want to perform it right now probably in my next upcoming videos i may also perform transformations scale basically i can perform standardization with respect to the data but since here mostly i'm using decision tree classifier random forest classifier i will not be doing over here because obviously decision tree and random forest will not be requiring that thing okay but if i'm using logistic regression i may go ahead with it okay so first of all i'm going to create a pipeline which says that the first thing is nothing but a decision tree classifier this decision tree classifier will be stored in this particular variable then i'm going to create this model which will be in the list format i'm going to write models dot append first is basically my decision tree the second is the thing that i'm going to create is basically random forge classifier this also i'm going to append it in inside my list of models and remember this all will be in the form of pipeline the third one is basically your svc i'm going to append it inside my models list okay so i have included three machine learning algorithm one is decision tree random forest and svc now after this we are going to define the voting and symbol okay so in the voting classifier you have the first parameter which is called as estimators inside this i will be giving the list of models okay whatever the list of models is basically added and the voting parameter will be having the hard value as uh the value itself right so inside this voting classifier you'll be able to see what all voting values can ha have over here okay so right now by default it is hard i'm going to take that particular value as hard now over here what i'm going to do next is that i'm going to just go and check it out and right now i'm seeing my model so this is basically our models the list of models and if i really want to check out my n symbol so n symbol is nothing but it will be the combination of all the models so you can see that voting classifiers of this all estimators okay and don't worry about the code guys this will be given in the github after this i'm going to perform repeated stratified k-fold for proper trade test splits the number of splits that i'm going to do is 10 and for each split i'm going to do the repetition of three so the total number of uh test will be 30 10 multiplied by 3 okay then i have the cross valve score in the cross valve score i'm going to take this ensemble model over here i'm going to give my data sets x and y the scoring parameter will be accuracy in case of imbalanced data set you can give roc auc curve that is roc underscore auc the cross validation cv will be nothing but it will be this repeated stratified fold and underscore jobs will be one so as soon as i execute it i will basically be getting all the scores from this ensemble techniques and over here once i see the end underscore scores these all values that you are seeing will be total number values will be 30 because i have done n underscore splits and with respect to this i've got this much right so 30 into 3 into 10 which is nothing but total number of iteration is 30 for each iteration i will be getting the voting classifier for this i'll be getting 94 96 90 to 90 and this count that you'll be doing right it will be for 30 repetitions because the number of splits are 10 the number of repetition is 3 for each split 3 repetitions so 10 into 3 will be nothing but 30. now if i go and find out the score mean okay n underscore scores dot mean probably it should work so it is nothing but 0.993 that basically means 93 percentage of the accuracy you can try out with different different things and remember one more thing guys in the pipeline when you're adding this kind of classifiers you can do along with the transformation and probably i've shown in one of my videos how to do transformation if you really want one more video i will in my next video i'll try to create this kind of custom and symbol techniques and also try to put up scale like whether i want to do any kind of transformation scaling or many more things right so inside this decision tree classify i can also give provides parameters i can also perform grid search cv i can also perform hyper parameter tuning and then i can take the combination of all this particular model and with the help of voting classifier i can find out the output now here i've just added three algorithms you can add 10 algorithms 12 algorithms 14 algorithm any number of algorithms suppose in this svc you only want to perform the transformation in svc you can perform it over here also in this you don't require because scaling and transformation will not be very much important but the main thing is the voting classifier right in ensembl techniques like xgboost or random forest this voting happens internally itself so in this video i've shown you how you can go ahead with your custom and symbol machine learning algorithms where you can combine multiple algorithms and test your problem statement so i hope you like this particular video please do subscribe the channel if you have not already subscribed and i'll see you all in the next video have a great day thank you and all bye