3 Main Types of Missing Data | Do THIS Before Handling Missing Values!

Key Takeaways

Hi, missing values can be a real headache. I always feel anxious when I see them. That's because you never know the actual value of the missing data point. Proper handling of missing data requires adequate knowledge of statistics and careful attention to detail. There are tons of methods for handling missing data, so how do you choose the right one? Remember, before looking into the methods for missing data handling, you need to understand why they disappeared. In this video, we will go through different types of missing data and provide examples of how to distinguish them. Missing data can occur in three main different types: missing completely at random, missing at random, and missing not at random. Missing completely at random is the best scenario. It occurs when the omitted observations are randomly missing, meaning the missingness is entirely unrelated to the observed data. For example, in a medical study, suppose the height measurements of patients are not observed because the measuring device occasionally malfunctioned randomly. In this case, missing values have nothing to do with patient characteristics like age, weight, health status, or the height variable itself. So, why is this type the most preferred? Because the missing value is totally random. Even if you decide to remove those observations, the remaining data is still a random unbiased subset of the entire sample. However, let's imagine another medical scenario where we have partially missing doctor visiting frequency data, which is influenced by the wealth or income of the patient. Poor patients can't afford to visit doctors frequently, so there are often missing values for low-income patients. This case is called missing at random. In other words, missing data is related to observed data or another variable. We have a higher likelihood or probability of missing doctor visiting frequency data for poor patients. Now, having the income data, we may be able to predict the frequency and fill in the missing values. The last type is missing not at random when the reason for the missingness is directly related to the value of the missing data itself. This creates a problem because you can't predict or explain it using other variables. For example, some wealthy people may not want to expose their income and simply don't record it. In this case, the missing income is related to the income variable itself. This may lead to underestimating the average of the income if those with higher incomes are the ones not reporting. If you just remove the missing points, you may lose the randomness in data since you accidentally removed all the wealthy people from the data set. Thus, you need to be super careful when dealing with not random missing data. We will refer to the methods for handling the missing values for all three types in future videos. So, I will recommend to follow us for more. If you want to learn more about artificial intelligence, subscribe to our channel to be aware of the new videos. Press the like button and let's discuss AI in the comments section.