Random Forest Tutorial | Why 100 Trees Beat 1 Tree

Key Takeaways

[Music] So in this video, we'll see how random forests improve performance over a single decision tree. A random forest combines many decision trees to make better predictions. Let's jump in and see how this forest grows. We're going to initialize a random forest classifier. Unlike a single decision tree, a random forest creates an ensemble of trees that work together. Our random forest has been trained with a 100 trees and it uses bootstrap sampling to create diverse training sets for each tree. The average tree in our forest has 5,526 nodes and a depth of 31.6. This complexity in individual trees is acceptable in a random forest because the ensemble approach helps prevent overfitting. Notice that the bootstrap sampling is enabled by default, which means that each tree is trained on a random subset of the data. This variation in training data helps make each tree in the forest unique, contributing to the model's robustness. We're now generating predictions on our test data using the random forest model. When generating predictions, the random forest takes votes from all 100 trees and determines the final prediction based on a majority vote. Our random forest model achieves an impressive 97% accuracy on the test data. Looking at the confusion matrix, we can see out of 27,453 non-diabetic patients, 27,287 were correctly classified as true negatives. 66 incorrectly classified as having diabetes or false positives. For diabetic patients, 1,759 correctly identified true positives. 788 were mclassified as non-diabetic false negatives. This gives us a high precision of 91% for the diabetes class, but a lower recall of 69%. Indicating that the model is more likely to miss some positive cases than to generate false alarms. The confusion matrix shows us clearly how well the model is identifying both positive and negative cases. Another advantage of random forest is they provide robust probability estimates. For each prediction, we get a probability score rather than just binary outcome. For example, patient number one did not have diabetes 0.9700 and diabetes 0.030. Patient 5 diabetes 0.910 0 for no diabetes. These probability scores give us a measure of confidence in each prediction. Important advantage of treebased models is their ability to provide feature important scores. Let's examine which features our random forest considers most important for diabetes prediction. Our random forest model identified these top five important features. HBA1C level 0.41 4130 blood glucose level 0.3228 BMI 0.1439 age 0105 and hypertension 0.0129 0129. The blue line shows the cumulative as we add features in order of importance. The red horizontal line indicates the 95% importance threshold. This is extremely valuable for model simplification. Rather than using all available features, we can focus on just the most important ones. We only need the top four features to capture 95% of the model's predictive power. This is valuable information for feature selection and model simplification. In a real world setting, we would potentially focus our data collection efforts on just these four measurements. HBA1C level, blood glucose level, BMI, and age. In this demonstration, we've seen how random forests provide a powerful improvement over decision trees while maintaining most of their benefits. [Music]