R Tutorial: Splitting criterion in trees

Key Takeaways

classification 3 uses a split condition to predict class labels based on one or more input variables the classification process starts from the root node of the tree and that each node the process will check whether the input value should recursively continue to the right or left sub branch according to the split condition this process stops when meeting any leaf or terminal nodes the idea behind classification trees is to split the data into subsets where each subset belongs to only one class this is accomplished by dividing this boot space into peer regions that is regions with samples from only one class with real data completely pure regions may not be possible so the decision tree will do the best it can to create regions that are as pure as possible boundaries separating these regions are called decision boundaries in the decision tree model makes classification decisions based on these decision boundaries in this diagram we have an example where we are able to partition the space into regions there are a hundred percent of peer meaning there are only examples from one class in each region the goal is to partition data a node into subsets that are as pure as possible in this example the partitions shown on the right results in more homogeneous subsets since these subsets contain more samples belonging to a single class than the result subsets shown on the left so the partition on the right results in peer subsets and it is the preferred partition therefore we need a way to measure the period of splits in order to compare different ways to partition a set of data it turns out that it works mathematically if we measure the impurity rather than purity of a split so the impurity measure of a note specifies how mixed the resulting subsets are since we want the results in subsets to have homogeneous class labels not mixed class labels we want the split that minimizes the impurity measure a common impurity measure used for determining the path split is the Gini index the lower the Gini index the higher the periods of the split so the decision tree will select the split that minimizes the Gini index besides the Gini index other impurity measures include entropy or information gay Mis classification rate but we we are not going to how to calculate each of these in this course all you need to know is that each one measure how makes the partitions are let's practice now how to compare models with a different split criteria er