Machine Learning Crash Course: Classification

Key Takeaways

you can think of model predictions as falling along a line between 0.0 and 1.0 where 0.01 means the model has 1% confidence that the given example falls in the positive class and 99% confidence that it falls in the negative class 0.5 means 50% confidence that the example Falls in either class 0.99 suggests the model has 9 9% confidence that it falls in the positive class and 1% confidence that it falls in the negative class by setting a classification threshold we tell the model at what confidence level to divide predictions into the positive and negative classes when the threshold is set at 0.5 all examples at 0.5 and above are classified as the positive class shown in purple while All Points below that threshold are classified as the negative class shown in yellow notice that several examples are wrongly classified we can track the number of correctly and incorrectly classified examples in a confusion Matrix the threshold chosen should be set based on the business purpose of the model because some mistakes are more costly than others as we increase the threshold notice that false positives or yellow examples wrongly classified as purple drop to zero if false positives are much more costly or risky than false negatives it makes sense sense to minimize them choosing 0.67 might be a good threshold in that case if the mistakes are roughly equal in cost a threshold between 0.49 and 0.56 might be a better choice here is a highly imbalanced data set what might be a good threshold value here if you set anywhere between 0.92 and 0.95 that's a good choice note that these visualizations have been simplified in reality model predictions are messy floating Point numbers the points will not be stacked but will fall all over the line overlapping with each other now try changing the classification threshold for different data sets for yourself