Python Tutorial : What is a decision tree?

Want to learn more? Take the full course at https://learn.datacamp.com/courses/extreme-gradient-boosting-with-xgboost at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- Because XGBoost is usually used with trees as base learners, we need to understand what an individual decision tree is, and how it works. Here is an example decision tree. As you can see, it has a single question that is being asked at each decision node, and only 2 possible choices, at the very bottom of each decision tree, there is a single possible decision. In this example decision tree for whether to purchase a vehicle, the first question you ask is whether it has been road-tested. If it hasn't, you immediately decide not to buy, otherwise, you continue asking questions, such as what the vehicle's mileage is, and if its age is old or recent. At the bottom, every possible decision will eventually lead to a choice, some taking many fewer questions to get to those choices than others. The concept of a base learner will be covered more extensively later, but for now, just think of any individual learning algorithm in an ensemble algorithm as a base learner. This is important because XGBoost itself is an ensemble learning method in that it uses the outputs of many models for a final prediction. Anyway, as you saw in the previous slide, a decision tree is a learning method that involves a tree-like graph to model either a continuous or categorical choice given some data. It is composed of a series of binary decisions (yes/no or true/false questions) that when answered in succession ultimately yield a prediction about the data at hand (these predictions happen at the leaves of the tree). Decision trees are constructed iteratively (that is, one binary decision at a time) until some stopping criterion is met (the depth of the tree reaches some pre-defined maximum value, for example). During construction, the tree is built one split at a