Python Tutorial: Classification-Tree Learning

Want to learn more? Take the full course at https://learn.datacamp.com/courses/machine-learning-with-tree-based-models-in-python at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- Welcome back! In this video, you'll examine how a classification-tree learns from data. Let's first start by defining some terms. A decision-tree is a data structure consisting of a hierarchy of individual units called nodes. A node is a point that involves either a question or a prediction. The root is the node at which the decision-tree starts growing. It has no parent node and involves a question that gives rise to 2 children nodes through two branches. An internal node is a node that has a parent. It also involves a question that gives rise to 2 children nodes. Finally, a node that has no children is called a leaf. A leaf has one parent node and involves no questions. It's where a prediction is made. Recall that when a classification tree is trained on a labeled dataset, the tree learns patterns from the features in such a way to produce the purest leaves. In other words, the tree is trained in such a way so that, in each leaf, one class-label is predominant. In the tree diagram shown here, consider the case where an instance traverses the tree to reach the leaf on the left. In this leaf, there are 257 instances classified as benign and 7 instances classified as malignant. As a result, the tree's prediction for this instance would be: 'benign'. In order to understand how a classification tree produces the purest leafs possible, let's first define the concept of information gain. The nodes of a classification tree are grown recursively; in other words, the obtention of an internal node or a leaf depends on the state of its predecessors. To produce the purest leafs possible, at each node, a tree asks a question involving one feature f and a split-point sp. But how does it know which feature and which split-point