Python Tutorial : Applying logistic regression and SVM

Key Takeaways

in this video we'll see how to run logistic regression and svm with scikit-learn the logistic regression class in scikit-learn is used just like the other models you've seen in the prerequisite course first we import logistic regression from scikit-learn you'll notice we're importing from linear model because logistic regression is a linear classifier more on this later then we instantiate an instance of the classifier we fit the classifier on our training set and then we can predict compute the score etc let's try this on an example data set in this case the wine classification data set built into scikit-learn we load the data set then we create and fit a logistic regression object we compute the training accuracy and see it's about 97% psych it learns logistic regression can also output confidence scores rather than hard or definite predictions let's do this with the predict problem and test it out on the first training example here the classifier is reporting over 99% confidence for the first class and very low probabilities for the other two as a reminder the little e means 10 ^ so you should interpret that first probability as nine point nine times 10 to the power of negative one or 0.99 or 99% will discuss these probabilities more in chapter 3 in scikit-learn the basic SVM classifier is called linear SVC for linear support vector classifier the linear SVC object works exactly the same way as logistic regression note that this data set has more than two classes psych it learns logistic regression and svm implementations handle this automatically we'll talk about how this works in Chapter three we can repeat these steps again for the SVC class which fits a non linear SVM by default as you can see the classifier achieves a hundred percent training accuracy this could be the classifier over fitting which is a risk we take when using more complex models like nonlinear svms later in this chapter we'll discuss what it means for a classifier to be linear or not by the way so far we've used the default hyper parameters for logistic regression linear SVC and SVC to remind you a hyper parameter is a choice about the model you make before fitting to the data and often controls the complexity of the model if the model is too simple it may be unable to capture the patterns in the data leading to low training accuracy this is called under fitting on the other hand if the model is too complex it may learn the peculiarities of your particular training set leading to lower test accuracy this is called overfitting this is a fundamental trade-off in machine learning in chapters 3 & 4 we'll delve into these classifiers in more detail so that by the end of the course you'll understand what many of the hyper parameters represent how they affect this fundamental trade-off and how to go about setting them now it's your turn to apply these classifiers