Python Tutorial: Dimensionality Reduction in Python | Intro

Want to learn more? Take the full course at https://learn.datacamp.com/courses/dimensionality-reduction-in-python at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- Hi there, my name is Jeroen Boeye and I'm a Machine Learning engineer at Faktion. In this course, I'll be teaching you how to reduce dimensionality in your datasets. Before we get going, it's important to clarify some concepts. When I mention dimensionality, I mean the number of columns you have in your dataset, assuming that you are working on a tidy dataset, such as the Pokemon dataset shown here. Working with a tidy dataset makes your life as a data scientist much easier. To do so, just make sure every column resembles a variable, or in other words feature, such as the properties of Pokemon like attack and defense in this example. Similarly, each row holds an observation for each variable. The first row gives us all the properties of one specific Pokemon. You end up with a value in each cell of your dataset. You can learn the number of rows and columns in a Pandas dataframe using the .shape attribute. For our small Pokemon dataset, .shape would return a tuple telling us there are five rows and seven columns. Throughout this course, we'll be using pandas a lot and we'll always pre-import it for you as 'pd'. When you have many columns in your dataset, say, more than 10, the data is considered high dimensional. And if you're new to that dataset, it could be hard to find the most important patterns because of the complexity that comes with high-dimensionality. To overcome this, we can reduce the number of columns using dimensionality reduction techniques. However, these techniques can also be useful for relatively low dimensional datasets, such as this one. In the dataset shown here, you'll notice that all Pokemon come from the same generation. If we're interested in how Pokemon are different, this feature would not be very useful