R Tutorial: What is cluster analysis?

Key Takeaways

hi my name is Dima and I'm very excited to have you join me in learning all about cluster analysis in our cluster analysis is a form of data exploration and the key to harnessing its power lies in understanding how it works so in this course you won't just learn the tools necessary to perform cluster analysis that's the easy part I will work with you to build the intuition behind the underlying methods but before we get to the how let's take a moment to discuss what is clustering no matter whether you're working with medical data retail data or sports data as a data scientist you are often presented with a bunch of data that you need to make sense of to understand what clustering is let's put aside the details of our data and instead focus on the toy example where the data is represented as a matrix containing the entries of cart suits so look at it another way this matrix is composed of rows containing our observations and columns tell us something that we measured across these observations we will refer to these columns as the features of our observations in cluster analysis we are interested in grouping our observations such that all members of a group are similar to one another and at the same time they are distinctly different from all members outside of this group imagine in this example we perform cluster analysis to find which observations are similar to one another based on what suit appears in each column in this case we identified three groups and color the observations accordingly to better see these patterns let's reorganize our observations into their respective colored clusters here we can start to see clear patterns that emerge fundamentally this is how cluster analysis works or to put it another way cluster analysis is a form of an exploratory data analysis or observations are divided into meaningful groups that share common characteristics amongst each other so what are the steps involved performing cluster analysis well first you must make sure that your data is ready for clustering meaning that your data does not have any missing values and that your features are in similar scales next you must decide on what metric is appropriate to capture the similarity between your observations using the features that you have once you have calculated this you can use a clustering method to group your observations based on how similar they are to each other into clusters but most importantly you will need to analyze the output of these clusters to determine whether they provide any meaningful insight into your data this often requires a deep understanding of the problem and the data that you are working with as you can see in this flowchart the analysis you perform on these clusters may require you to iterate on the clustering steps until you converge on a meaningful grouping of your data the first three chapters of this course will help you unpack this process in this chapter you will gain a deeper understanding of what it means for two observations to be similar or more specifically dissimilar you will also learn why the features of your data need to be comparable to one another in chapters 2 & 3 you will learn how to use two commonly used clustering methods hierarchical clustering and k-means clustering at the end of these chapters and in Chapter four you will work through two case studies where clustering analysis provides a unique perspective into the underlying data so let's begin