R Tutorial: Testing the extremes with Grubbs' test

Want to learn more? Take the full course at https://campus.datacamp.com/courses/anomaly-detection-in-r at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. -- In this lesson, you'll learn a statistical procedure called Grubbs' test, which can help in assessing whether the data contain outliers. A visual assessment for outliers like a boxplot can work really well when the majority of the data are close together, and a few outliers are clearly separated. This boxplot shows the temperature data from the previous video. It's reasonable to think that the point circled in red lies close enough to the majority of the data that we can't be certain that it's an outlier. When this happens, we can use a statistical test called Grubbs' test to make sure. Grubbs' test assesses whether the point that lies farthest from the mean in a dataset could be an outlier. This point will either be the largest or smallest value in the data. Grubbs' test works by assuming that the data are normally distributed and it is therefore important to first ensure that this assumption is plausible for the data you're analyzing, before proceeding to use the test. A histogram provides a common way to check the normality assumption visually. In R, a histogram is produced using the hist function. The main argument of the hist function is the data to show, while the breaks argument controls how many bins the histogram has. For larger datasets, breaks can be increased to get a more detailed view of the distribution. When checking for normality, we should be aware of both the symmetry and shape of the histogram. Normally distributed data, should have an approximately symmetrical and bell-shaped histogram, which is roughly true for the temperature data. If the data distribution seems lop-sided or has more than a single peak, then the Grubbs' test should not be used. If you come across data like this, that's ok, you'll learn other techniques late