5.4 Intro to Scikit-learn (L05: Machine Learning with Scikit-Learn)

Key Takeaways

yeah so in this relatively short video i want to introduce machine learning with psychic learn you have seen psychic learn before in the k-nearest neighbor lectures where i was working with the k-nearest neighbor classifier from second learn however here i want to take a step back and introduce the library properly and then after this short video there will be a longer video where i will go over some of the aspects for preparing the training data set using psychic learn tools that make that more convenient and more efficient than compared to what we have done before and lastly the last video then we'll again go over some of yeah the cool concepts in scikit-learn where we can combine like the pre-processing and machine learning classifier fitting and training and stuff like that using scikit-learn pipelines okay but let's briefly talk about machine learning with scikit-learn now so we are right now here yeah personally i always refer to psychic learn as the main machine learning library for python and that is because it's yeah a relatively established library it has a very large user base it's really nice to use and yeah it's just a great library and as far as i know it's also the most widely used machine learning library so one thing to note though is that scikit-learn is not not for deep learning so for deep learning we have other libraries and deep learning is a separate topic that i'm teaching in stats 453 so here we're not talking about the deep learning capabilities it's just a machine learning library however it's probably one of the best designed libraries i know so i also usually refer to scikit-learn as a showcase library for implementing yeah a good api and a good python library in general in python because what's really great about um scikit-learn is that it's very consistent and has a very well thought out way of using it everything is compatible to each other and um yeah you will rarely find any bugs in this library and so forth so yeah it's it's a great library um and it's also already a relatively old so it was initially released in 2007 it was like 13 years ago it's a long time ago however that doesn't mean that it's like um dated it's still very much up to date a lot of people are contributing it to it so originally it started as a summer or google sum of code project by um david um corner po and later on other um contributors joined so later on lots of people started contributing so i was just checking on github there are more than 1875 contributors now for psychic learn and it's used by almost 150 000 people so it's a huge library there are lots of efforts that go into it and you can really see that based on the quality of their code so yeah here's the link to the official website where you find the documentation and tutorials and i think this is maybe yeah the one main paper if you use that later on in your research project would be nice to get to cite the library so um that's just a common convention in academia if you use the software library don't forget to cite it because people put a lot of work into it and it would be yeah also nice for them if you cite their library yeah so here is an overview of the scikit-learn estimator api so with estimator api i mean apis for regressors like regression analysis classes and classifiers so we call them in scikit-learn regressor regressors and classifiers these are the main types of classes for supervised learning so this is um for supervised learning and here this doesn't actually exist so this is just some code that explains the main concepts of the estimator api that goes back to the object oriented programming paradigm that i explained in the last video so the main aspects of any classifier or regressor are listed here here try to abstract that a little bit so what you uh how the super based estimator usually works is that you initialize it you're using certain hyper parameters and the hyperwriters they get assigned in the constructor here so this is usually you don't see that part you just provide the high parameters and there's some thing going on in the init method and that will yeah create the object so when you have something like uh let's say my new classifier and you initialize a cane neighbors classifier then i'm running out of space so i'm writing this on a new line there will be hyper parameters like n neighbors you can set it for example to three and stuff like that there are usually multiple hyperventilators that you can set so that is one aspect of of course the supervised estimator api and the hypermeters they get assigned here in the initial initialization call um a very important method is the fit method so in the fit method you you fit the object before you can use it so you for example after initializing it call classifier dot um fit on using the features so x is the design matrix so these are the features so it's um and the shape is n training examples times uh n features or you can also say m features so we don't use the same letter twice and then we have also a label array so this is an array it also has the shape and train so this is just a vector basically y is a vector and then during the fitting um there will be certain attributes also assigned to the estimator again recall from the last video on the object oriented programming paradigms i explained that in psychic learn these fit attributes so things that get created during or after calling fit are having this underscore here this trailing underscore so there are additional attributes available only after fitting and these are the ones with the trailing underscore often you don't have to use them sometimes we will be using them in the class but if you see something like an attribute with a training underscore it means that it is an attribute created during model fitting yeah then there's the predict method that you can use after calling fit so you would then do classifier dot predict and here you can have let's say a new um data set that has the same shape as your train data set so the same features but it could be let's say new examples that you haven't encountered during training it could be for example the test set so you can do predictions then um and there's also a score method so the score method it depends on whether we have a classifier or a regressor in the case of a classifier the score method is just a convenience method for computing the accuracy the classification accuracy and for regressors it's computing the um are r2 sq the r squared so the coefficient of determination yeah lastly there are the private methods the ones with the leading underscore and these are the ones in practice you don't have to worry about they mainly just exist to make the implementation of the code more efficient and more readable related to concepts of via code refactoring so private methods you can kind of ignore they are usually only used internally and yeah that is the main outline of a supervised estimator where we have the fit method the predict method the score method and then sometimes also additional methods but these are the main ones yeah just as another way of looking at it here is a flow chart outlining the scikit-learn estimator api or mainly the workflow so after initializing an estimator we would call fit it's the first step calling fit or using the training data so this one will fit fit the model so we have then our fitted model and then after we fitted the model we can use the test data set or any other new data set that has the same features as the training data to make predictions so then we can call predict and we get back the labels so that's just the main outline of how we can use an estimator here is an estimator in action again so here i'm using the k nearest neighbors classifier from the scikit-learn neighbors sub-module so second learn organizes their code into sub-modules because otherwise it would be kind of hard to maintain that library so to make it a little bit more readable so here we import the k-nearest neighbor classifier and then i have a convenience function and an extend for plotting decision regions so what we are doing here is we are first initializing an estimator and then after we initialize it we can fit it to the training data like outlined in the previous slide note that i'm only using two features here the feature the third feature and the fourth feature in the iris data set so this is the iris data set by the way and that is just for i would say educational purposes because you can only plot the decision regions for two dimensions because it's just the way of how we visualize things because we can't really visualize higher dimensional um scatter plots right so we can maybe visualize a 3d scatter plot but everything beyond three dimensions would be hard to visualize and even making a 3d plot on a 2 d slide here would be kind of challenging for visual purposes so here we are just using two features just for educational purposes in practice of course you rarely have a data set with two features any case so after fitting we are calling this plot decision regions function um to visualize that and this is how it would look like so here we have a decision region plot of the k nearest neighbor classifier fit to the petal length and petal width and note that the plot decision regions function internally would call on predict to make the predictions basically yeah and what you can see here is that the k-nearest neighbor classifier with three neighbors um i would say gets 100 accuracy here on this training set so it's maybe even overfitting a little bit here so what you can see is it can nicely separate out the blue squares from the triangles and circles so i think uh the blue squares these were satosa red was rosy color and the green one was virginica flowers so you can see separating of organica and versicolor is a little bit more tricky here it's actually doing a good job with separating or catching this one however one might argue this is a little bit overfitting so it could be um if you have new flowers that flowers this region might be actually veginica flowers however we will talk more about overfitting in later lectures so yeah this was it a brief introduction to second learn in the next video we will be taking a look at how we prepare the training set so basically we we will take a look at how we get x train and wide train from the iris data set um including techniques like normalizing the data and pre-processing the data and stuff like that so yeah and with that um this video was just a short teaser using second learn in the next video we will go into a little bit more detail