Python Tutorial : How do we measure success?

Want to learn more? Take the full course at https://learn.datacamp.com/courses/case-study-school-budgeting-with-machine-learning-in-python at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- The next step is to decide how we decide if our algorithm works. Choosing how to evaluate your machine learning model is one of the most important decisions an analyst makes. The decision balances the real-world use of the algorithm, the mathematical properties of the evaluation function, and the interpretability of the measure. Often we hear the question "how accurate is your model?" Accuracy is a simple measure that tells us what percentage of rows we got right. However, sometimes accuracy doesn't tell the whole story. Consider the case of identifying spam emails. Let's say that only 1% of the emails I receive are spam. The other 99% are legitimate emails. I can build a classifier that is 99% accurate just by assuming every message is legitimate, and never marking any message as spam. But this model isn't useful at all because every message, even the spam, ends up in my inbox. The metric we use for this problem is called log loss. Log loss is what is generally called a "loss function," and it is a measure of error. We want our error to be as small as possible, which is the opposite of a metric like accuracy, where we want to maximize the value. Let's look at how logloss is calculated. It takes the actual value, 1 or 0, and it takes our prediction, which is a probability between 0 and 1. The greek letter sigma (which looks like an uppercase E below) indicates that we're taking the sum of the logloss measures for each row of the dataset. We then multiply this sum by -1 over N, the number of rows, to get a single value for loss. We will unpack this math a little more by looking at an example. Consider the case where the true label is 0, but we predict confidently that the label is 1. In this case, because y is 0