Heart Disease Detection in Python - Machine Learning Project

Key Takeaways

what is going on guys welcome back in this video today we're going to train and evaluate multiple machine learning models for heart disease detection so let us get right into it not [Music] AED all right so we're going to do heart disease detection in Python today and for this we're going to train multiple machine learning models on the same data set we're going to compare them in terms of their performance and then we're going to try to maximize this performance by doing Hy of parameter tuning and in the end we're also going to take a look at the feature importances so we're going to analyze which of these features down below are particularly important when it comes to detecting whether someone has a heart disease or not now this data set is a little bit special because usually when I do these machine learning projects or machine learning processes um I use data sets where we have to do a lot of pre-processing so we have some features that are textual uh or categorical that need to be turned into numerical features this particular data set here already has only numerical features so we can see that we only have uh numerical values here even if we display all the features we can see that all of them are basically numbers so either binary or numeric uh even the categorical ones and because of that we don't really need to do any pre-processing here we can just go ahead and uh use the data right away of course you can do feature Engineering in terms of creating new features if you want to you can maybe also also think about uh taking something like chest pain type here and splitting it into multiple columns so basically doing one H en coding you can do that if you want to but today we're going to focus mainly on training comparing evaluating uh and also analyzing the feature importances so you can see we have a bunch of features here that are quite uh interesting chest pain type blood pressure uh maximum heart rate and so on and then we also have this target which is uh a binary feature basically or a binary Target zero for no disease and one for disease and this is what we're trying to predict here so that is our classification task so to say now the column names can be a little bit um difficult to understand CP stands for chest pain um call stands for cholesterol uh then we have uh T rest BPS stands for resting blood pressure so if you don't know what a feature means just go to this kaggle page and look it up in this box up here um other than that you don't really need to know what you're working with while you're working with it except for knowing what kind of data it is categorical binary uh numeric and so on so you will find a link to this data set in the description down below just download it extract it and then you can open up a jupyter notebook and load this CSV file using pandas uh speaking of that we're going to also install a couple of packages so open up your terminal and use pip to install pandas matte plot lip caborn I think we're also going to use numpy and of course we're going to use pyit dlearn these are the packages that we're going to need in this video today make sure you have them installed and once you have this we can start by importing pandas as PD and then we're going to say the data set is going to be PD read CSV hard CSV and we can take a look at this you can see again all of these features are numerical features we can use them right away for some models we will have to do some scaling because some models are are scale sensitive others are not but what we're going to do now is we're going to just take this data right away turn it into X Andy and then do a train test split so we're going to say from SK learn. model selection import train test spit then we're going to say the X and Y data is going to be the data when we drop the target is going to be VX data so the input data and Y is going to be just the target uh and then of course the train test split is going to be XT train X test y train y test that is going to be splitting X and Y and we're going to choose the test size which is quite high today today because this data set um let's say the models do quite well if you just use 20% uh for the testing set so if you use 80% for training the accuracy and everything is going to be quite high so we're going to make it a little bit more difficult by providing less data so I'm going to go uh with a test size of 40% so if this was a different data set you would want to go with something like uh 20% for test size but in this case we're going to train on 60% of the data and evaluate on 40% of the data and we're also going to provide a random State nine for neural 9 just to make sure we have the same results every time we run this because otherwise of course it's random so X train X test y train y uh y test uh and then we're going to start with the scale insensitive models so some of the models don't really care about scale so insensitive um one such model would be a random Forest classifier another one would be naive BAS another one would be uh gradient boosting so we're going to just go ahead and say from SK learn. Ensemble we're going to start with a random Forest classifier we're going to say forest is equal to random forest classifier and we're going to train this random Forest classifier on X train and on y train so now we're going to repeat this process for the different model types I'm going to just copy and paste this I'm going to replace this by a goshan naif base classifier so I'm going to say NB clf is going to be naive base or goshan naive base NB clf fit same as before and then we're going to also say I think it's still Ensemble import what was it gradient boost and classifier GB clf I think this one should be scale independent usually you only have scale dependent models when they're caring about distances that are like ukian geometric distances um not sure if that's entirely true what I'm saying right now but I think that gradient boosting should be scale and sensitive uh unless there's some distance measure in there but I think it shouldn't be um so we're going to say gradient boost and classifier we're going to say GB clf we're going to train and now we have three models trained on the data we're going to evaluate them in a second but let's also add three more that are scale sensitive so I'm going to say scale sensitive models and we're going to train a simple kers neighbors classifier uh but for this of course we need to First scale the data so we're going to say from SK learn do uh pre-processing we're going to import the standard scaler we're going to create an instance of it and we're going to um scale our training data so we're going to say xtrain is going to be equal to scaler fit transforms so we're going to fit the scaler on the data and we're going to also transform the data and of course this should result not in xtrain but in xtrain scaled and the same thing will be done for the test data but we're not going to fit again for the test data we're going to um we're going to just transform there you go and now we're going to say from sklearn do Neighbors not naive base neighbors import K neighbors classifier KN andn is going to be equal to K neighbors classifier we're going to always use the default type of parameters and we're then going to tune the model that performs best so K&N is going to be K neighbors classifier uh K&N is going to be fit it onto X train scaled and Y train why don't we have to train uh why don't we have to scale the uh Y data because of course K andn and all the other um models are only focusing on the input and the output doesn't really matter the output is just the output it's just the value that the model produces in the case of K nearest neighbors classification it's just the color of the dot the number that's associated with the class in our case it's binary so it's Z or one we don't don't really need to scale that it doesn't make a difference we only need to scale the training data so the input data um yeah not the training data the input data we also need to do to uh scale the test data the input of the test data uh so we fit the model here then we copy this we paste it and we're going to also train uh a logistic regression so linear model or yeah logistic regression even though it's called regression it's actually a classif ification so we're going to call this lock logistic regression lock fit and then finally we're going to train a support Vector machine a support vector classifier and there you go so we have these uh six models now and we're going to evaluate them first of all in terms of accuracy that's quite simple all we have to to do is we have to say dot score and then X test and Y test so like this the forest performs quite well and we're going to do the same thing now for the other models we're going to do for the naive base classifier the same thing for the gradient boost the same thing then we're going to uh do a slightly different thing for the K&N because here we're going to say X test scaled y test we're going to then copy and paste this so we're going to say lock we're going to say SVC and we can see actually that the forest performs the best and the gradient boost than the support Vector classifier so in terms of accuracy this one is the best now you could say right now we're working with medical data and I'm not a medical professional I don't know if that's correct and how they make the decisions but my opinion is or my my feeling is that it's not so bad if I say someone might have a heart disease and then they are healthy that's not that much of a big deal it's way more problematic if I say hey someone's healthy and then they have a heart problem or heart disease so recall is more important in my opinion than Precision now for those of you who don't know what recall is and don't know what Precision is accuracy just tells us how often am I right about my prediction Precision tells me when I say someone has a heart disease how often am I right about this this is not the same so accuracy says I make guesses let's say I make 100 guesses so this is accuracy um I make 100 guesses and out of these 100 guesses I uh have let's say uh 80 correct that would be an accuracy of 80% or in our numbers up here 0.8 now Precision is different Precision is not how often how many guesses do I have Precision would be I don't make a 100 guesses but let's say I make 200 guesses doesn't really matter but I say 100 times that the patient has a disease how often when I say that they have a disease not how often when I say something in general how often when I say they have a disease am I correct about this um if I have a Precision of let's say 20% so 20 times I'm actually correct about this my Precision is 20% recall asks a different question let's say that there are actually 100 patients patients with heart disease recall asks how many of them do I find so if I say that everyone every single one of the patients has a heart disease my precision so if I say 200 have disease but actually only 100 actually have one my Precision is 50% but my recall is 100% because I can find all the patients with heart disease so if I always say yes of course that's not intelligent but the recall is the important thing because I want to spot every patient with a heart disease this is at least my reasoning I don't know if that's what what doctors think but this would be my reasoning that recall is way more important I want to find all the patients with heart disease uh it's okay if I say someone has a heart disease and then they don't have a heart disease because you know still a human expert could uh take a look at it but it's better if I give you a false positive than a false negative this is my reasoning so we're going to see how well the models perform in terms of that we're going to say from sklearn do metric import the recall score and we're going to say that at the predictions of the forest model are going to be making predictions on the test data and then we're going to say that Forest recall score is going to be recall score of the actual results and the predictions so in this case we would have a recall of actually no it's not the same but it's quite similar to the um to the accuracy we're going to take this now we're going to copy it three times actually I should only copy two times but we're going to change this now to be [Music] equal to what was the other one na base classifier n b then I'm going to change this to gradient boost and classifier GB and we can see that um the recall for the gradient boosting is actually the same as the one for the forest so if we care about this they are actually uh the same but of course the forest has more accuracy so probably uh it has higher precision as well and now we can go ahead and copy this and do the same thing for K andn but this time with the scale data the rest stays the same name and now we can copy this twice we can change this to log we can change this to lock and finally we have SVC and we can change this to SVC so for the recall either the forest or the gradient boosting classifier seems to be the best choice and because of that we're going to focus on the forest from now on we want to know how uh good we can get by using the forest now it's not a guarantee by the way that hyperparameter tuning will necessarily improve the model now I'm not sure actually uh or actually I could be sure if I look at the results I'm not going to do it right now um but I'm not sure if our forest is going to uh necessarily become better by hyperparameter tuning uh actually I wanted to use for the forest as well a random state so I'm going to say random state is equal to 9 and run all of this again I think it stays the same um but the idea is that hyperparameter tuning can have a good effect if I maybe have more data if my model maybe or if the data is more complicated I don't know but it's not a guarantee that hyperparameter tuning will produce the best model because of course we're doing cross validation or maybe we're using a validation set but we're not actually working with the test data so we're evaluating the model on the test data but we're doing either cross validation or uh we're using a validation set to do the hyperparameter tuning which means that we're we're optimizing for different data so we don't have a guarantee for this but before we do that I want to also plot another metric which is the RO which uh I forgot what it stands for uh but we have the RO and the area under the curve this is a plot that plots the uh false positive rate on the x- axis and the true positive rate on the y axis so basically the recall on the y- AIS and then also the false positive rate on so how often I misclassify something as positive uh even though it's negative and this curve shows us the trade-off between uh these two metrics so we're going to say import matplot lip s or matplot lip P plot as PLT from SK learn we want to also from SK learn metrics we want to import uh the ROC curve and the ROC Au score area under the curve and uh what we're going to do here now is we're going to say give me the probabilities so the forest can give us uh the probabilities not just the classifications I think that a cane neighbors classifier cannot do that I think K&N does not have a uh predict prop a function oh actually it does so which one does not have it does the SBC have it it actually has it didn't think so so maybe all the models that we trained uh have that so the logistic one definitely has it yeah maybe maybe they all have it uh but we're going to get the probabilities from the forest by calling the predict prop a function and we're going to say predict um based on the test data make predictions and then we're going to plot this curve so we're going to get the false positive rate and the true positive rate uh and the thresholds by calling the ROC curve function onto y test and the Y probabilities so basically the idea is we get some probabilities and we say okay after a certain probability I'm going to classify this as a disease as a yes uh and below that I'm going to say no and the question now is is uh what threshold leads to what kind of fpr and tpr actually not trp tpr uh true positive rate and false positive rate and we're going to plot this trade-off so we're going to say PLT plot FBR tpr pltx label we're going to say on the x-axis we have the false positive rate on the Y AIS we have the true positive rate also called recall and the title is going to be just Roc curve I'm going to say PLT show and this is going to give us nothing because there's a problem what is the problem y test y probabilities oh yeah we need to say want to have this just a second uh Second Dimension here so that is the curve in this case it doesn't look too uh interesting because the model performs quite well we can also see that by calculating the area under the curve which is like a discrete interval of everything below this area one is perfect and I think we're going to get something pretty close so y test uh y props this gives us almost one 0.99 um but yeah this is basically the trade-off maybe if we want to talk about this we can use a different model maybe we can go with lock and see that this looks different now and this has a way worse uh area under the curve but the basic idea is I want to maximize the recall but I also don't want to you know destroy my uh Precision so this is basically the opposite of precision how often am I wrong about my positive guesses and how many of the positive instances do I find this is a tradeoff and in order to get for example to 100% to find all of them using this not so good model I would have to basically classify everything as um as a disease because only then can I get 100% but I can get a pretty good performance already by just missing 65% uh or something like this of the uh of the negative instances so this is not a good model you can see the trade-off is not very good but I can maybe change this and do this yeah SVC doesn't have predict probability okay so this was the mod that doesn't have it uh KNN does KNN have it yes so for KNN I can get um not so good recall around 80% but I also miss quite a lot so this is also not a good uh model to do that if I go with the gradient boost classifier uh in order to get 100% recall so the to find all the diseases uh I would have to allow for something like 22% of false positive rate which means 22% of the time when I say something is actually a disease it's actually not a disease uh yeah so this is also pretty good model but the best one seems to be uh the forest so we're going to keep working with the forest and for that we're going to do hyperparameter tuning using grid search so we're going to open up a new section here we're going to say hyperparameter tuning and we're going to say from sklearn do model selection import git search CB the parameter grit is going to be equal to a bunch of different hyperparameters these come of course from the model so go to the documentation pyed learn documentation for random forest classifier and then you will find uh some hyper parameters in my case now I'm going to just copy and paste them from my um from my prepared code because what we have here is we have the number of estimators we have the max depth we have the Min sample split basically this is the number of trees in a forest a random Forest is just a collection of decision trees so we have different number of decision trees we have a limiting depth so saying that the tree cannot have more than a certain depth so so none is unlimited 10 20 30 uh then we have Min sample split uh how many samples we need to split a node Min samples leave and then we also have Max features which limits the amount of features so these are just some hypop parameters we can now say that we want to have a random forest model so a random Forest classifier we're going to set end jobs equal to -1 to parallelize all of this and we're going to add the random state of nine again and then we're going to say G G search is equal to uh grid search CV we're going to pass the model we're going to pass the parameter grit and we're going to say we want to do a three-fold cross validation what this basically means is we take the data we split it into three folds we use two for training one for testing and we do that for all the combinations so maybe to show this visually uh this would be all okay this would be all of my data I make a split like this I use these two for training and I evaluate them on this one I evaluate the model on this one then I use uh these two for training and I evaluate the model on this one and then I do uh training on these two and I evaluate the model on this one that's what uh three-fold cross validation is so for every combination of parameters we're going to do it three times uh so if you you have a large data set you want to reduce this you want to keep this smaller this training Works quite uh quickly so we're going to just yeah easily uh do a bunch of iterations so finally what I want to do also is I want to say n jobs1 here because I want to parallelize this process as well and when I run this or actually not when I run this but when I say grid search. fit and now I'm going to use X test uh X train sorry X train and Y train I'm going to do all of this for multiple models we can actually go ahead and say verbos equals 2 to get some more information you can see we're fitting three folds for each of 324 candidates totaling 972 fits why is this um why are these the numbers because we have um One Forest is going to have 100 estimators no limit two here one here and square root the next one is going to have 100 none 2 1 log two 100 none 2 1 none then 200 um or actually 100 none 2 two square root so all of these have to be multiplied um and while this is training we can actually do that um so we can actually open up do we have a calculator gome calculator um how much do we have here we have 3 * 4 * 3 * 3 * 3 that is 324 and we're doing three-fold cross validation so 972 uh so this seems to be done right now and as a result we get the optimal classifier so we can say best Forest is equal to grit search and then best estimator so we can take a look at the best forest and it has 500 estimators and nothing else that is uh the best result so all the others are kept at default this is what it means when they're not listed here random State 9 is of course fixed and jobs negative 1 is also fixed but that is the only thing that we change now it would make more sense now to comment out all of this and see if we can get an even better model by providing for example uh 600 700 and so on so I can just run this and um this would result now in still 500 so it seems to be a very good number we're going to keep that and we're going to now do a feature importance analysis so we're going to open up a new section here we're going to call it feature importances and we're going to say that the feature importances actually for this we're going to use numpy SMP we're going to say that the feature importances are equal to Best Forest feature importances underscore uh the features so the feature names are best Forest feature names in underscore um and the underscore is just a convention and Cy could learned that these are things that are the result of training they're not um there by default they have to be calculated so to say we're going to say sorted index is equal to NP Arc sort going to sort the feature importances um so that we know which indices are uh which are the indices so that the whole thing is sorted and then we're going to say sorted features is going to be equal to Features sorted indices or sorted index and then we're going to say sorted importances is going to be equal to feature importances Ed index um often times what I also do in these tutorials is I zip and then I do so I basically do sorted zip and then I provide key is equal to uh X1 Lambda expression X1 is also uh one way to do it but we can do it like this as well uh which is maybe easier to understand for some people uh and then we can choose colors for the bars because we're going to visualize to using map uplip we're going to say the colors are PLT color map uh yellow and green or yellow to Green sorted importances divided by the maximum of the sorted importances so basically we want to have the most extreme color for the largest value and that's basically just scaling it and now we're going to say PLT bar horizontal so bar H sorted features on the x-axis sorted importances on the y- AIS um or actually no sorted features is going to be on the y axis because we have a horizontal bar sorted importances is going to be on the x-axis and the color is going to be colors then we're going to say pltx label feature [Music] importance the Y label is going to be features and we're going to say PLT show and maybe also title feature importances so when I show this we can see that we have certain features being way more important than other features now before we Analyze This I want to also take a look at correlations before we do any random Forest classification so in the data itself we might find some correlations and I think that this is going to be a good learning that this doesn't have to be exactly the same as the feature importances so so I can import caborn s SNS and I can say that in a data frame I can have correlation so if I say DF correlation I will get this table here and I can just say SNS heat map of this correlation Matrix um I want to have The annotation equal to true so I want to see the values inside of the heat map I want to also have the same color map as before which is yellow green um and I want to say PLT figure figure size is going to be equal to 1210 and then we can see that these are the correlations um before we do any training so this is just how the data correlates and the target correlates very strongly you need to also consider negative correlations because negative correlation doesn't mean it doesn't correlate it just means it correlates uh the other way around so it's an indirect uh proportionality uh to maybe make this clearer let's go and apply the apps function on to the correlation we're not really interested in how they correlate but only the magnitude and then you can see that the strongest correlation between the target variable and uh the other features is between uh xang and old Peak so if we go to the data what is this this is the column 1 2 3 4 5 6 7 8 9 10 so I think 9 and and 10 uh exercise induced angena and then we have ST depression induced by exercise relative to rest um I'm not really sure what these mean but these two features seem to be um if you just look at the correlation the most important ones and then we have the chest pain type and we have the what is that um maximum heart rate achieved so these are also important but when we look at the feature importances we can see that the chest pain type and the what was it again uh the maximum heart rate achieved are more important than old Peak and definitely more important than the exercise induced in ginaa so this doesn't seem to be important at all for the for the random Forest classifier but it has a very strong correlation so so you can see that this is not always necessarily uh equal a strong correlation doesn't mean that it's necessarily important for the decision- making when it comes to class spe ifying uh whether someone has a heart disease or not so that would be my conclusion um actually maybe one more thing because we didn't evaluate this let's see if our best Forest uh actually outperforms the other Forest so let's do a score X test and Y test and we get 9853 what did we get before we got 536 yeah it's the same it has the exact same uh score score let's see if it also has the same Recall why predictions is going to be best Forest uh or actually let's go for the for the ROC so let's just what happened now let's just go and do this but with the best Forest just interested in that uh yeah looks the same and now maybe Also let's do the recall so let's go and copy that and let's do it here as well I think this is going to be equal right now and maybe this is just because the data is too simple and we don't have a lot of data best Forest yeah it seems 9859 what did we have here 9859 okay so it has the exact same performance which doesn't have to always be the case sometimes it can be worse some times it can be better but yeah the most important feature when it comes to predicting whether someone has a heart disease or not is the type of chest pain so not if someone has chest pain but the type of chest pain we have four values for that um and then also the maximum heart rate achieved and then CA what is CA uh CA is the slope of the peak exercise ST segment whatever that is so I'm not a medical professional I don't know what that is but yeah this is an interesting analysis our model performs quite well of course what you would do now if you want to use this which I would not recommend this is just a coding tutorial but if you want to use this you could just take the model best forest and do predictions on new data so you just take new data and you make predictions on the new data you just have to format it in the same way just pass the numbers in the correct order and then you're going to get um either the prediction or you can say give me the probability that someone has a hard dis disease and then this works as well so that's it for today's video I hope you enjoyed it and hope you learned something if so let me know by hitting a like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you in the next video and bye