Python Tutorial: Decision-Tree for Classification

hi my name is le koala I'm a data scientist and I'll be your instructor in this course you'll be learning about tree based models for classification and regression in Chapter one you'll be introduced to a set of supervised learning models known as classification and regression tree or cart in Chapter two you'll understand the notions of bias-variance tradeoff and mobile and assembly chapter three introduces you to bagging and random forests chapter four deals with boosting specifically with adaboost and gradient boosting finally in Chapter five you'll understand how to get the most out of your models through hyper parameter tuning given a labeled data set a classification tree learns a sequence of affairs questions about individual features in order to infer the labels in contrast to linear models trees are able to capture nonlinear relationships between features and labels in addition trees don't require the features to be on the same scale through standardization for example to understand trees more concretely we'll try to predict whether a tumor is a malignant or benign and the wisconsin' breast cancer data set using only two features the figure here shows a scatter plot of two cancer cell features with malignant tumors in blue and benign tumors in red when a classification tree is trained on this data set the tree learns a sequence of fell's questions with each question involving one feature and one split point take a look at the tree diagram here at the top the tree asks whether the concave points mean of an instance is smaller or equal to zero point zero 51 if it is the instance traverses the true branch otherwise it traverses the false branch similarly the instance keeps traversing the internal branches until it reaches an end the label the instance is then predicted to be that a bit of the prevailing class at that end the maximum number of branches separating the top from an extreme end is known as the maximum depth which is equal to two here now that you know what a classification tree is less fit one with scikit-learn first import decision tree classifier from eskalene tree as shown in line one also import the functions train test plate from SK learn model selection and accuracy score from SK learn metrics in order to obtain an unbiased estimate of our models performance you must evaluate it on an unseen test set to do so first split the data into 80% train and 20% test using treinta split set the parameter stratified to y in order for the train and test sets to have the same proportion of class labels as the unsplit data set you can now use decision tree classifier to instantiate a tree classifier DT with a maximum depth of two by setting the parameter max depth two to note that the parameter random set is set to 1 for reproducibility then call the fit method on DT and pass X train and Y train to predict the labels of the test set called the predict method on DT finally print the accuracy of the test set using accuracy score to understand the trees predictions more concretely let's see how it classifies instances in the feature space a classification model divides the feature space into regions where all instances in one region are assigned to only one class label these regions are known as decision regions decision regions are separated by surfaces called decision boundaries the figure here shows the decision regions of a linear classifier note that the boundary is a straight line in contrast as here on the right a classification tree produces rectangular decision regions in the feature space this happens because at each split made by the tree only one feature is involved now let's practice