R Tutorial: Handling missing data

Want to learn more? Take the full course at https://learn.datacamp.com/courses/practicing-machine-learning-interview-questions-in-r at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- In this video we are going to answer two fundamental questions around dealing with missing values in your data. An important question is what to do with your missing values. First, explore your dataset, identify its missingness and visualize it. Then talk to your client about these missing values and see if there is any business rationale that can be applied to deal with them. In general, there are three main avenues: * Ignore them by discarding samples/variables with high levels of missingness. Use this option sparingly as this often leads to loss of valuable data. * Impute them, i.e., replace them with other (hopefully more meaningful) values. * Accept them and proceed to choose methods that naturally deal with these missing values. Unfortunately, not many methods can do that. The strategy depends on the type of missingness you have. The Maniar package provides many useful functions to identify, visualize and deal with missing data. any_na will tell you if there are missing values in your dataframe. Sometimes you need to manually replace one or more missing data symbols with NAs, as done here. You can easily summarize the level of missingness across variables and instances in your dataset. Here we see that 5 out of 6 variables have missing values. Visualizing the missing values in a dataframe is very easy. Just invoke the vis_miss function. You can optionally arrange the rows according to their missingness with cluster=TRUE. The gg_miss_case function displays the missingness at the row level or cases. In this example, only a very small fraction of the observations have two or more missing values. There exist 3 types of missing data: * Missing Completely at Random (MCAR) * Missing at Random (MAR) * Missing Not